1 Star 0 Fork 0

Nuclei-Software / npk-tinymaix

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

TinyMaix

中文 | English

TinyMaix是面向单片机的超轻量级的神经网络推理库,即TinyML推理库,可以让你在任意单片机上运行轻量级深度学习模型~
我们的设计原则:易用性 > 移植性 > 速度 > 空间

tinyML基础介绍: TinyML
查看已移植的芯片测试结果: benchmark
好消息: TinyMaix有奖移植

关键特性

  • 核心代码少于400行(tm_layers.c+tm_model.c+arch_cpu.h), 代码段(.text)少于3KB
  • 低内存消耗,甚至Arduino ATmega328 (32KB Flash, 2KB Ram) 都能基于TinyMaix跑mnist(手写数字识别)
  • 支持INT8/FP32/FP16模型,实验性地支持FP8模型,支持keras h5或tflite模型转换
  • 支持多种芯片架构的专用指令优化: ARM SIMD/NEON/MVEI,RV32P, RV64V
  • 友好的用户接口,只需要load/run模型~
  • 支持全静态的内存配置(无需malloc)
  • MaixHub 在线模型训练支持

在Arduino ATmega328上运行mnist demo

mnist demo
0000000000000000000000000000
0000000000000000000000000000
0000000000000000000000000000
000000000077AFF9500000000000
000000000AFFFFFFD10000000000
00000000AFFFD8BFF70000000000
00000003FFD2000CF80000000000
00000004FD10007FF40000000000
00000000110000DFF40000000000
00000000000007FFC00000000000
0000000000004FFE300000000000
0000000000008FF9000000000000
00000000000BFF90000000000000
00000000001EFE20000000000000
0000000000CFF800000000000000
0000000004FFB000000000000000
000000001CFF8000000000000000
000000008FFA0000000000000000
00000000FFF10000000000000000
00000000FFF21111000112999900
00000000FFFFFFFFA8AFFFFFFF70
00000000AFFFFFFFFFFFFFFA7730
0000000007777AFFF97720000000
0000000000000000000000000000
0000000000000000000000000000
0000000000000000000000000000
0000000000000000000000000000
0000000000000000000000000000
===use 49912us
0: 0
1: 0
2: 89
3: 0
4: 1
5: 6
6: 1
7: 0
8: 0
9: 0
### Predict output is: Number 2, prob=89

TODO

  1. 将 tm_layers.c 优化到 tm_layers_O1.c, 目标提升速度到 1.4~2.0X 已完成
  2. 针对64/128/256/512KB内存限制,找到合适的骨干网络
  3. 增加例程: Detector,KWS,HAR,Gesture,OCR,...
  4. ...

如果想参与进TinyMaix的开发,或者想与TinyML爱好者交流,
请加入我们的telegram交流群:https://t.me/tinymaix

TinyMaix 设计思路

TinyMaix 是专为低资源的单片机设计的AI神经网络推理框架,通常被称为TinyML

现在已经有很多TinyML推理库,比如TFLite micro, microTVM, NNoM, 那为什么我们又捏了TinyMaix这个轮子呢?

TinyMaix是两个周末业余时间完成的项目,所以它足够简单,可以再30分钟内走读完代码,可以帮助TinyML新手理解它是怎么运行的。

TinyMaix希望成为一个足够简单的TinyML推理库,所以它放弃了很多特性,并且没有使用很多现成的NN加速库,比如CMSIS-NN

在这个设计思路下,TinyMaix只需要5个文件即可编译~

我们希望TinyMaix可以帮助任何单片机运行AI神经网络模型, 并且每个人都能移植TinyMaix到自己的硬件平台上~

注意:虽然TinyMaix支持多架构加速,但是它仍然需要更多工作来平衡速度和尺寸

设计内的特性

  • 最高支持到mobilenet v1, RepVGG的骨干网络
    • 因为它们对单片机来说是最常用的,最高效的结构
    • 基础的 Conv2d, dwConv2d, FC, Relu/Relu6/Softmax, GAP, Reshape
    • MaxPool, AvgPool (现在使用stride代替)
  • FP32浮点模型, INT8量化模型, FP16半精度模型
  • 转换keras h5或tflite到tmdl
    • 简单模型使用keras/tf训练已经足够
    • 复用了tflite现成的量化功能
  • 模型统计功能
    • 可选以减少代码尺寸

可考虑添加的特性

  • INT16量化模型
    • 优点:
      • 更精确
      • 对于 SIMD/RV32P 指令加速更友好
    • 缺点:
      • 占用了2倍的FLASH/RAM
  • Concat算子
    • 优点:
      • 支持mobilenet v2, 模型精度更高
    • 缺点:
      • 占用了2倍的RAM
      • concat 张量占用了更多时间,使得模型运算变慢
      • 需要更多转换脚本工作转换分支模型到扁平结构
  • Winograd卷积优化
    • 优点:
      • 可能加速卷积计算
    • 缺点:
      • 增加了RAM空间和带宽消耗
      • 增大了代码段(.text)尺寸
      • 需要很多变换,弱单片机可能会消耗更多时间

不考虑添加的特性

  • BF16模型
    • 多数单片机不支持BF16计算
    • 精度不会比INT16高太多
    • 占用了2倍的FLASH/RAM
  • AVX/vulkan 加速
    • TinyMaix是为单片机设计的,而不是为强大的电脑/手机设计
  • 其他多样化的算子
    • TinyMaix仅为单片机提供基础模型算子支持,如果你需要更特殊的算子,可以选择TFlite-micro/TVM/NCNN...

例程体验

mnist

MNIST是手写数字识别任务,足够简单以至于可以在ATmega328这样的8位单片机上运行。
在电脑上测试:

cd examples/mnist
mkdir build
cd build 
cmake ..
make
./mnist

mbnet

mbnet (mobilenet v1) 是适用于移动手机设备的简单图像分类模型,不过对单片机来说也稍微沉重了些。
例程里的模型是mobilenet v1 0.25,输入128x128x3的RGB图像,输出1000分类的预测 它需要至少128KB SRAM 和 512KB Flash, STM32F411是典型可以运行该模型的最低配置。

在PC上测试运行mobilenet 1000分类图片例程

cd examples/mbnet
mkdir build
cd build 
cmake ..
make
./mbnet

如何使用 (API)

加载模型

tm_err_t tm_load (tm_mdl_t* mdl, const uint8_t* bin, uint8_tbuf, tm_cb_t cb, tm_mat_t in);

mdl: 模型句柄;
bin: 模型bin内容;
buf: 中间结果的主缓存;如果NULL,则内部自动malloc申请;否则使用提供的缓存地址 cb: 网络层回调函数;
in: 返回输入张量,包含输入缓存地址 //可以忽略之,如果你使用自己的静态输入缓存

移除模型

void tm_unload(tm_mdl_t* mdl);

输入数据预处理

tm_err_t tm_preprocess(tm_mdl_t* mdl, tm_pp_t pp_type, tm_mat_t* in, tm_mat_t* out);
TMPP_FP2INT //用户自己的浮点缓存转换到int8缓存 TMPP_UINT2INT //典型uint8原地转换到int8数据;int16则需要额外缓存 TMPP_UINT2FP01 //uint8转换到01的浮点数 u8/255.0
TMPP_UINT2FPN11//uint8转换到-1
1的浮点数

运行模型

tm_err_t tm_run (tm_mdl_t* mdl, tm_mat_t* in, tm_mat_t* out);

如何移植

TinyMaix的核心文件只有这5个:tm_model.c, tm_layers.c, tinymaix.h, tm_port.h, arch_xxx.h

如果你使用没有任何指令加速的普通单片机,选择 arch_cpu.h, 否则选择对应架构的头文件

然后你需要编辑tm_port.h,填写你需要的配置,所有配置宏后面都有注释说明

注意 TM_MAX_CSIZE,TM_MAX_KSIZE,TM_MAX_KCSIZE 会占用静态缓存。

最后你只需要把他们放进你的工程里编译~

怎样训练/转换模型

在examples/mnist下有训练脚本可以学习如何训练基础的mnist模型

注意:你需要先安装TensorFlow (>=2.7) 环境.

完成训练并保存h5模型后,你可以使用以下脚本转换原始模型到tmdl或者c头文件。

  1. h5_to_tflite.py
    转换h5模型到浮点或者int8量化的tflite模型 python3 h5_to_tflite.py h5/mnist.h5 tflite/mnist_f.tflite 0
    python3 h5_to_tflite.py h5/mnist.h5 tflite/mnist_q.tflite 1 quant_img_mnist/ 0to1
  2. tflite2tmdl.py
    转换tflite文件到tmdl或者c头文件
    python3 tflite2tmdl.py tflite/mnist_q.tflite tmdl/mnist_q.tmdl int8 1 28,28,1 10
================ pack model head ================
mdl_type   =0
out_deq    =1
input_cnt  =1
output_cnt =1
layer_cnt  =6
buf_size   =1464
sub_size   =0
in_dims    = [3, 28, 28, 1]
out_dims   = [1, 1, 1, 10]
================   pack layers   ================
CONV_2D
    [3, 28, 28, 1] [3, 13, 13, 4]
    in_oft:0, size:784;  out_oft:784, size:680
    padding valid
    layer_size=152
CONV_2D
    [3, 13, 13, 4] [3, 6, 6, 8]
    in_oft:784, size:680;  out_oft:0, size:288
    padding valid
    layer_size=432
CONV_2D
    [3, 6, 6, 8] [3, 2, 2, 16]
    in_oft:0, size:288;  out_oft:1400, size:64
    padding valid
    layer_size=1360
MEAN
    [3, 2, 2, 16] [1, 1, 1, 16]
    in_oft:1400, size:64;  out_oft:0, size:16
    layer_size=48
FULLY_CONNECTED
    [1, 1, 1, 16] [1, 1, 1, 10]
    in_oft:0, size:16;  out_oft:1448, size:16
    layer_size=304
SOFTMAX
    [1, 1, 1, 10] [1, 1, 1, 10]
    OUTPUT!
    in_oft:1448, size:16;  out_oft:0, size:56
    layer_size=48
================    pack done!   ================
    model  size 2.4KB (2408 B) FLASH
    buffer size 1.4KB (1464 B) RAM
    single layer mode subbuff size 1.4KB (64+1360=1424 B) RAM
Saved to tmdl/mnist_q.tmdl, tmdl/mnist_q.h

现在你有了tmdl或者C头文件,把它放到你的工程里编译吧~

使用 Maixhub 在线训练模型

使用 MaixHub, 你可以下载或者上传已有模型, 同时可以在线训练你的 AI 模型, 不需要编写训练代码,也不需要机器学习基础,点点鼠标模型就训练好啦!

  • 注册 MaixHub 并登录。
  • 你可以从模型库下载已有模型,或者上传分享你的模型。
  • 创建一个在线模型训练项目, 采集数据集并在线训练,最后会得到:
    • .tmdl文件和.h文件,用其中一个即可。
    • report.json, 训练报告信息, json 格式, 我们可以在此文件中获取到 标签(labels)或者 anchors, 在代码中可能会用到的参数,注意每次训练参数都不一样,注意更改,否则无法正确识别
  • 有两种项目类别,分类 和 检测, 第一次使用建议使用 分类
  • 训练是有多个骨干网络(backbone)可选择,根据你的单片机的内存大小选择合适的骨干网络,内存越小则需要选择更小的骨干网络。
  • 为了更好地理解 MaixHub 的使用方法, 开始可以不选择 tinymaix 平台, 而是选择 tfjs 平台训练,训练出可以在手机上运行的模型进行体验。
  • examples 目录下找到对应的例程,比如 maixhub_image_classification 或者 maixhub_image_detection 来运行你训练的模型

怎样添加新平台的加速代码

对于新增平台,你只需要在src里添加arch_xxx.h文件并实现其中的函数即可,主要为以下几个函数(重要性降序排列,不重要的函数可以直接拷贝纯CPU运算的函数):

a. TM_INLINE void tm_dot_prod(mtype_t* sptr, mtype_t* kptr,uint32_t size, sumtype_t* result)
	实现平台相关的点积函数,可以使用MAC相关的加速指令加速。

b. TM_INLINE void tm_dot_prod_pack2(mtype_t* sptr, mtype_t* kptr, uint32_t size, sumtype_t* result)
	实现平台相关的双通道点积函数。(仅提供到双通道是因为有些芯片平台的寄存器不足以支持更多通道的点积加速)

c. TM_INLINE void tm_postprocess_sum(int n, sumtype_t* sums, btype_t* bs, int act, mtype_t* outp, sctype_t* scales, sctype_t out_s, zptype_t out_zp)
	实现平台相关的批量后处理函数,注意n为2的次幂。

d. TM_INLINE void tm_dot_prod_3x3x1(mtype_t* sptr, mtype_t* kptr, sumtype_t* result)
	实现平台相关的3x3点积加速
e. TM_INLINE void tm_dot_prod_gap_3x3x1(mtype_t* sptr, mtype_t* kptr, uint32_t* k_oft, sumtype_t* result)
	实现平台相关的3x3 gap的点积加速

贡献/联系

如果你需要向TinyMaix贡献代码,请先阅读“TinyMaix设计思路”一节,我们只需要“设计内的特性”和“可考虑添加的特性”。

如果你想要提交你的移植测试结果,请提交到 benchmark.md. 我们非常欢迎你移植TinyMaix到自己的芯片/板子上,这会证明使用TinyMaix运行深度学习模型是非常容易的事情~

如果你对TinyMaix的使用和移植有问题,可以在此仓库提交Issues。

如果你有商业或私有项目咨询,你可以发邮件到support@sipeed.comzepan@sipeed.com (泽畔).

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

暂无描述 展开 收起
C 等 3 种语言
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
1
https://gitee.com/Nuclei-Software/npk-tinymaix.git
git@gitee.com:Nuclei-Software/npk-tinymaix.git
Nuclei-Software
npk-tinymaix
npk-tinymaix
main

搜索帮助