# rknn-yolo **Repository Path**: zcxnll_admin/rknn-yolo ## Basic Information - **Project Name**: rknn-yolo - **Description**: Rock5a中ffmpeg cpu解码 yoloV5 和 V8 推理；Rock5a是Radxa（瑞莎）推出，使用了瑞芯微 RK3588S 处理器。 - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 3 - **Created**: 2024-01-03 - **Last Updated**: 2024-01-03 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # rknn-yolo #### 介绍 Rock5a中ffmpeg cpu解码 yoloV5 和 V8 推理；Rock5a是Radxa（瑞莎）推出，使用了瑞芯微 RK3588S 处理器。 #### 软件架构 软件架构说明 ### rock5a 安装 参考 ``` export PATH="/home/rock/workspace/rknpu2/runtime/RK3588/Linux/rknn_server/aarch64/usr/bin:$PATH" export LD_LIBRARY_PATH=/home/rock/workspace/rknpu2/runtime/RK3588/Linux/librknn_api/aarch64:$LD_LIBRARY_PATH export PATH=/home/nyy/workspace/pcl/vtkinstall/include:$PATH https://blog.csdn.net/m0_55217834/article/details/130583886?spm=1001.2101.3001.6650.5&utm_medium=distribute.pc_relevant.none-task-blog-2%7Edefault%7ECTRLIST%7ERate-5-130583886-blog-130205729.235%5Ev38%5Epc_relevant_anti_t3_base&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2%7Edefault%7ECTRLIST%7ERate-5-130583886-blog-130205729.235%5Ev38%5Epc_relevant_anti_t3_base&utm_relevant_index=6 https://www.zhihu.com/column/c_1654749776326512640 http://www.hbzwlsw.com/guocanju/fubeiderongyao/2-29.html http://www.hlm543.top/index.php/vod/play/id/Z4dCCS/sid/1/nid/1.html ``` ## rock5a 安装 ``` https://wiki.radxa.com/Rock5/downloads https://github.com/radxa-build/rock-5a/releases ubuntu 中下使用 balenaEtcher-1.14.3-x64.AppImage 制作系统盘 官方ubuntu系统： rock-5a_ubuntu_jammy_kde_b18.img.xz ``` ![](./picture/rock5a-iso.png) ### rock5a 安装 ``` sudo apt-get update sudo apt-get install openssh-server ``` ### 软件包 filezilla-server 需要重新安装，但是我无法找到相应的安装文件 ``` sudo rm -rf /var/lib/dpkg/info/filezilla-server* sudo dpkg --remove --force-remove-reinstreq filezilla-server sudo apt install filezilla-server sudo apt-get install openssh-server ``` ## rknn-toolkit2 安装 ### python 虚拟环境 ``` #创建 virtualenv环境 sudo apt install virtualenv #安装virtualenv软件 virtualenv -p /usr/bin/python3.7 venv #创建虚拟环境 source venv/bin/activate #激活venv环境 (venv) firefly@firefly:~$ pip3 -V #查看当前pip3所在Python的路径 pip 21.0.1 from /home/firefly/venv/lib/python3.7/site-packages/pip (python 3.7)pip ``` ## rock5a中安装 ### 安装相关依赖包（numpy、h5py &opencv） ``` sudo apt-get update sudo apt-get install cmake gcc g++ libprotobuf-dev protobuf-compiler sudo apt-get install liblapack-dev libjpeg-dev zlib1g-dev #pip3 install --upgrade pip #更新pip包的版本 #pip3 install wheel setuptools #安装 Python 打包工具 sudo apt-get install net-tools ``` ### rknpu2 环境安装 ``` cd rknpu2 sudo cp ./runtime/RK3588/Linux/librknn_api/aarch64/* /usr/lib sudo cp ./runtime/RK3588/Linux/rknn_server/aarch64/usr/bin/* /usr/bin/ ``` ### rknpu2 进入 /home/rock/workspace/rknn-toolkit2-master/doc ``` pip install -r requirements_cp310-1.5.2.txt -i https://pypi.tuna.tsinghua.edu.cn/simple/ ``` ## PC机中安装 ### ubuntu22.04 中安装 RKNN2-1.5.2 ``` 1、安装 pip install -r requirements_cp38-1.5.2.txt -i https://pypi.tuna.tsinghua.edu.cn/simple 2、安装 pip install rknn_toolkit2-1.5.2+b642f30c-cp38-cp38-linux_x86_64.whl -i https://mirrors.aliyun.com/pypi/simple/ ``` ### pt--onnx--rknn 转换 (网上常见的修改yolo.py 中class Detect(nn.Module): 的 def forward(self, x): ) ``` def forward(self, x): z = [] # inference output for i in range(self.nl): x[i] = self.m[i](x[i]) # conv return x 经过测试yolov5-v4 到 yolov5-v7 都在可以转换成功 v4版本： python ./models/export.py --weights yolov5s.pt --img 640 --batch-size 1 v7版本： python export.py --weights yolov5s.pt --data data/coco128.yaml --include onnx --opset 12 --batch-size 1 至此转换为 onnx 在 RKNN2-1.5.2 中 rknn-toolkit2-master/examples/onnx/yolov5 的 test.py（onnx转rknn） 无法转换上边的 onnx 模型， 因上边的 onnx中是sigmoid 激活函数；RKNN2-1.5.2 的 test.py（onnx转rknn）只支持 relu激活函数； 解决：用 RKNN2-1.4.0 中的test.py（onnx转rknn）可以把以上的onnx转换成功。 在安装RKNN2-1.5.2 后 执行 python test-1.4.py ``` ### 在rock5a中测试 ``` 需要在rknpu2-1.4.0 版本中才能推理成功；初步分析原因是新版本rknpu为了适配转onnx不再作修改（修改yolo.py 中class Detect(nn.Module): 的 def forward(self, x)）， 导致与老板版不兼容 /home/rock/workspace/rknpu2-1.4.0/examples/rknn_yolov5_demo 中 sudo bash ./build-linux_RK3588.sh cd ./install/rknn_yolov5_demo_Linux/ ./rknn_yolov5_demo ./model/RK3588/yolov5s-v7.rknn ./model/bus.jpg ./rknn_yolov5_demo ./model/RK3588/yolov5s-640-640.rknn ./model/bus.jpg ``` ### pt--onnx--rknn yolov5转换 (官方给出的 支持yolov5-yolov8 及 yolovX ) ``` 下载 airockchip 官方中的 yolov5 https://github.com/airockchip/yolov5.git 此时的版本是是airockchip 在yolov5-6.2版本基础上优化的专门适用于RK3588系列芯片 下载后PC机中安装pytorch环境： 把yolov5s.pt 转换为：yolov5s.torchscript.pt ：python export.py --rknpu rk3588 --weight yolov5s.pt python export.py --rknpu rk3588 --weight yolov5s-ulv62.pt 下载：rknn_model_zoo ：https://github.com/airockchip/rknn_model_zoo.git 使用安装 RKNN2-1.5.2 的python环境即可 cd ./rknn_model_zoo/models/CV/object_detection/yolo/RKNN_model_convert yolov5s.torchscript.pt 转换为 yolov5s-rkv62.rknn模型： ：python ../../../../../common/rknn_converter/rknn_convert.py --yml_path ./yolo.yml ``` ### 在rock5a中测试 ``` 下载：rknn_model_zoo ：https://github.com/airockchip/rknn_model_zoo.git cd ./rknn_model_zoo/libs/rklibs 根据 reademe提示下载： # RK3566/RK3568/RK3588/RV1106/RV1103 NPU 依赖库 git clone https://github.com/rockchip-linux/rknpu2 # RGA调用依赖库，不区分硬件平台 git clone https://github.com/airockchip/librga cd ./rknn_model_zoo/models/CV/object_detection/yolo/RKNN_C_demo/RKNN_toolkit_2/rknn_yolo_demo bash ./build-android_RK3588.sh #编译c++ demo cd ./install/rk3588/Linux/rknn_yolo_demo 把yolov5s-rkv62.rknn模型 拷贝到此处 ./rknn_yolo_demo v5 fp yolov5s-rkv62.rknn ./model/bus.jpg 结果如下图： ``` ![](./picture/rock5a_testyolov5.png) ### pt--onnx--rknn yolov8转换 (官方给出的 支持yolov5-yolov8 及 yolovX ) ``` 下载 airockchip 官方中的 yolov5 https://github.com/airockchip/yolov5.git 此时的版本是是airockchip 在yolov5-6.2版本基础上优化的专门适用于RK3588系列芯片 下载后PC机中安装pytorch环境： 把yolov5s.pt 转换为：yolov5s.torchscript yolo export model=yolov8s-ul.pt format=rknn 下载：rknn_model_zoo ：https://github.com/airockchip/rknn_model_zoo.git 使用安装 RKNN2-1.5.2 的python环境即可 cd ./rknn_model_zoo/models/CV/object_detection/yolo/RKNN_model_convert torchscript 转换为 rknn模型： ：python ../../../../../common/rknn_converter/rknn_convert.py --yml_path ./yolo.yml ``` ## 在rock5a中安装ffmpeg ``` git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg/ # Install necessary packages. sudo apt-get install build-essential yasm cmake libtool libc6 libc6-dev unzip wget libnuma1 libnuma-dev cd ffmpeg # Configure ./configure --enable-nonfree --enable-gpl --enable-libx264 --enable-cuda-nvcc --enable-libnpp --extra-cflags=-I/usr/local/cuda/include \ --extra-ldflags=-L/usr/local/cuda/lib64 --disable-static --enable-shared ./configure --prefix=/usr/local/ffmpeg --disable-static --disable-stripping --disable-doc \ --enable-shared --enable-nonfree --enable-cuda --enable-gpl --enable-libx264 --enable-cuvid --enable-nonfree --enable-cuda-nvcc --enable-libnpp --extra-cflags=-I/usr/local/cuda-11.8/include --extra-ldflags=-L/usr/local/cuda-11.8/lib64 sudo ./configure --prefix=/usr/local/ffmpeg4.4 --disable-static --disable-stripping --disable-doc \ --enable-shared --enable-nonfree --enable-cuda --enable-gpl --enable-libx264 --enable-cuvid --enable-nonfree --enable-cuda-nvcc --enable-libnpp --extra-cflags=-I/usr/local/cuda-11.8/include --extra-ldflags=-L/usr/local/cuda-11.8/lib64 sudo ./configure --prefix=/usr/local/ffmpeg --enable-swscale --enable-swresample --enable-gpl --enable-shared make clean make -j45 sudo make install export PATH=$PATH:/usr/local/ffmpeg/bin export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/ffmpeg/lib export PKG_CONFIG_PATH=/usr/local/ffmpeg/lib/pkgconfig export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/rock/workspace/video_detection/log4cplus/lib sudo nohup ./run.sh & ls -l | grep "^-" | wc -l ``` ## 在rock5a中 开机自启动配置 ``` sudo ln -fs /lib/systemd/system/rc-local.service /etc/systemd/system/rc-local.service sudo vim /etc/systemd/system/rc-local.service //添加 [Install] WantedBy=multi-user.target Alias=rc-local.service sudo touch /etc/rc.local sudo chmod 777 /etc/rc.local sudo vim /etc/rc.local #!/bin/bash date -s '2021-12-08 13:45:00' ``` ## opencv 安装 ``` apt-get install build-essential libgtk2.0-dev libavcodec-dev libavformat-dev libjpeg-dev libswscale-dev libtiff5-dev apt-get install libgtk2.0-dev apt-get install pkg-config apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev tar -axvf opencv-3.4.20.tar.gz cd ./opencv-3.4.20 sudo mkdir build cd ./build cmake -D CMAKE_BUILD_TYPE=RELEASE -D OPENCV_GENERATE_PKGCONFIG=ON -DBUILD_EXAMPLES=ON -DBUILD_DOCS=OFF -DBUILD_PERF_TESTS=OFF -DBUILD_TESTS=OFF -D BUILD_opencv_world=ON -D OPENCV_DNN_CUDA=OFF -D WITH_CUDA=OFF -D CMAKE_INSTALL_PREFIX=/home/rock/workspace/opencv/opencv3420 .. sudo make -j45 ``` ### log4cpus ``` cd ./log4cplus-2.0.8/ mkdir build cmake -DCMAKE_INSTALL_PREFIX=/home/rock/workspace/video_detection/log4cplus/libx64 .. cmake --build . --config release -j8 cmake --install . ``` /etc/init.d/ntpd start /etc/init.d/ntpd stop /etc/init.d/ntpd restart rtsp://admin:nyy123456@192.168.1.168:554 ./configure --prefix=/home/rock/workspace/video_detection/log4cplus ### cpu ``` 2. 获取当前CPU支持的频点 sudo cat /sys/devices/system/cpu/cpufreq/policy6/scaling_available_frequencies 408000 600000 816000 1008000 1200000 1416000 1608000 1800000 2016000 2208000 2304000 3. 获取cpu运行的模式 sudo cat /sys/devices/system/cpu/cpufreq/policy6/scaling_available_governors conservative ondemand userspace powersave performance schedutil 默认是自动变频模式：schedutil（恢复的话设置为该模式即可） 4. 设置手动定频模式：userspace echo userspace > /sys/devices/system/cpu/cpufreq/policy6/scaling_governor 5. 设置频率为2016000 sudo echo 2016000 > /sys/devices/system/cpu/cpufreq/policy6/scaling_setspeed 确认是否设置成功 sudo cat /sys/devices/system/cpu/cpufreq/policy6/cpuinfo_cur_freq 2016000 ``` ### NPU ``` 2. 获取NPU支持的频点 cat /sys/class/devfreq/fdab0000.npu/available_frequencies 300000000 400000000 500000000 600000000 700000000 800000000 900000000 1000000000 3. 获取NPU运行的模式 cat /sys/class/devfreq/fdab0000.npu/available_governors userspace powersave performance simple_ondemand 默认是自动变频模式：simple_ondemand（恢复的话设置为该模式即可） 4. 设置手动定频模式：userspace echo userspace > /sys/class/devfreq/fdab0000.npu/governor 5. 设置频率为1000000000 echo 1000000000 > /sys/class/devfreq/fdab0000.npu/userspace/set_freq sudo cat /sys/class/devfreq/fdab0000.npu/cur_freq 6. 查看NPU的负载 sudo cat /sys/kernel/debug/rknpu/load ```