From 4e3f62e0f59fae7845915bcfe2ea1eb5fc8ca74e Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Fri, 11 Oct 2024 07:05:17 +0000 Subject: [PATCH 01/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/BertTextClassification/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/BertTextClassification/README.md | 222 ----------------------- 1 file changed, 222 deletions(-) delete mode 100644 contrib/BertTextClassification/README.md diff --git a/contrib/BertTextClassification/README.md b/contrib/BertTextClassification/README.md deleted file mode 100644 index 877a73156..000000000 --- a/contrib/BertTextClassification/README.md +++ /dev/null @@ -1,222 +0,0 @@ -# 文本分类 - -## 1. 介绍 - -文本分类插件基于 MindXSDK 开发，在晟腾芯片上进行文本分类，将分类结果保存。输入一段新闻，可以判断该新闻属于哪个类别。 -该模型支持5个新闻类别：体育、健康、军事、教育、汽车。 - -### 1.1 支持的产品 - -本项目以昇腾Atlas310卡为主要的硬件平台。 - -### 1.2 支持的版本 - -支持的SDK版本为2.0.4。 -支持的CANN版本为5.0.4。 - -### 1.3 软件方案介绍 - -基于MindX SDK的文本分类业务流程为：待分类文本通过预处理，将文本根据字典vocab.txt进行编码，组成numpy形式的向量，将向量通过 appsrc 插件输入，然后由模型推理插件mxpi_tensorinfer得到每种类别的得分，再通过后处理插件mxpi_classpostprocessor将模型输出的结果处理，最后得到该文本的类别。本系统的各模块及功能描述如表1.1所示： - - -表1.1 系统方案各子系统功能描述： - -| 序号 | 子系统 | 功能描述 | -| ---- | ------ | ------------ | -| 1 | 文本输入 | 读取输入文本 | -| 2 | 文本编码 | 根据字典对输入文本编码 | -| 3 | 模型推理 | 对文本编码后的张量进行推理 | -| 5 | 后处理 | 从模型推理结果中寻找对应的分类标签 | -| 7 | 保存结果 | 将分类结果保存文件| - -### 1.4 代码目录结构与说明 - -本工程名称为文本分类，工程目录如下图所示： - -``` -. -│ build.sh -│ README.md -│ tree.txt -│ -├─mxBase -│ │ build.sh -│ │ CMakeLists.txt -│ │ main.cpp -│ │ -│ ├─BertClassification -│ │ BertClassification.cpp -│ │ BertClassification.h -│ │ -│ ├─data -│ │ vocab.txt -│ │ -│ ├─model -│ │ bert_text_classification_labels.names -│ │ -│ ├─out -│ │ prediction_label.txt -│ │ -│ └─test -│ Test.cpp -│ Test.h -│ -└─sdk - │ build.sh - │ flowChart.png - │ main.py - │ run.sh - │ tokenizer.py - │ - ├─config - │ bert_text_classification_aipp_tf.cfg - │ bert_text_classification_labels.names - │ - ├─data - │ vocab.txt - │ - ├─model - │ bert_text_classification_aipp_tf.cfg - │ bert_text_classification_labels.names - │ model_conversion.sh - │ - ├─out - │ prediction_label.txt - │ - ├─pipeline - │ BertTextClassification.pipeline - │ - └─test - test.py - test.sh - test_input.py -``` -### 1.5 技术实现流程图 - -![image](sdk/flowChart.png) - - -## 2 环境依赖 - -推荐系统为ubuntu 18.04，环境依赖软件和版本如下表： - -| 软件名称 | 版本 | -| -------- | ------ | -| cmake | 3.10.2 | -| mxVision | 2.0.4 | -| python | 3.9.2 | - -确保环境中正确安装mxVision SDK。 - -在编译运行项目前，需要设置环境变量： - -``` -export MX_SDK_HOME=${SDK安装路径}/mxVision -export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:/usr/local/Ascend/driver/lib64:${LD_LIBRARY_PATH} -export PYTHONPATH=${MX_SDK_HOME}/python:${PYTHONPATH} - -export install_path=/usr/local/Ascend/ascend-toolkit/latest -export PATH=/usr/local/python3.9.2/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH -export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH -export ASCEND_OPP_PATH=${install_path}/opp -``` - -- 环境变量介绍 - -``` -MX_SDK_HOME：MindX SDK mxVision的根安装路径，用于包含MindX SDK提供的所有库和头文件。 -LD_LIBRARY_PATH：提供了MindX SDK已开发的插件和相关的库信息。 -install_path：ascend-toolkit的安装路径。 -PATH：添加python的执行路径和atc转换工具的执行路径。 -LD_LIBRARY_PATH：添加ascend-toolkit和MindX SDK提供的库目录路径。 -ASCEND_OPP_PATH：atc转换工具需要的目录。 -``` - -## 3 模型转换 - -**步骤1** 请参考https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/bert_text_classification.pb -下载模型的pb文件，存放到开发环境普通用户下的任意目录，例如：$HOME/models/bert_text_classification。 - -**步骤2** 执行以下命令使用atc命令进行模型转换： - -cd $HOME/models/bert_text_classification - -atc --model=bert_text_classification.pb --framework=3 --input_format="ND" --output=bert_text_classification --input_shape="input_1:1,300;input_2:1,300" --out_nodes=dense_1/Softmax:0 --soc_version=Ascend310 --op_select_implmode="high_precision" - -**步骤3** 执行以下命令将转换好的模型复制到项目中model文件夹中： - -``` -cp ./bert_text_classification.om $HOME/sdk/model/ -cp ./bert_text_classification.om $HOME/mxBase/model/ -``` - -**步骤4** 执行成功后终端输出为： - -``` -ATC start working now, please wait for a moment. -ATC run success, welcome to the next use. -``` - -## 4 编译与运行 - -**步骤1** 从https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip下载测试数据并解压，解压后的sample.txt和test.csv文件放在项目的mxBase/data和sdk/data目录下。 - -**步骤2** 按照第 2 小节 环境依赖 中的步骤设置环境变量。 - -**步骤3** 按照第 3 小节 模型转换 中的步骤获得 om 模型文件。 - -**步骤4** 将本项目代码的文件路径中出现的 ${SDK目录} 替换成自己SDK的存放目录，下面是需要替换的代码。 - -``` -mxBase目录下的CMakeList.txt中的第13行代码： -set(MX_SDK_HOME ${SDK目录}) - -sdk/pipeline目录下BertTextClassification.pipeline文件中的第26行： -"postProcessLibPath": "${SDK目录}/lib/modelpostprocessors/libresnet50postprocess.so" -``` - -**步骤5** pipeline项目运行在sdk目录下执行命令： - -``` -python3 main.py -``` - -命令执行成功后在out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 - -**步骤6** mxBase项目在mxBase目录中，执行以下代码进行编译。 - -``` -mkdir build -cd build -cmake .. -make -``` - -编译完成后，将可执行文件 mxBase_text_classification 移动到mxBase目录下，执行下面代码运行 - -``` -./mxBase_text_classification ./data/sample.txt -``` - -执行成功后在服务器的mxBase/out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 - -## 5 精度测试 - -**步骤1** 已按照上一小节 编译与运行 的步骤将样例运行成功。 - -**步骤2** 下载[数据集](https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip)后解压，将解压后的test.csv文件分别放在sdk/data目录和mxBase/data目录。 - -**步骤3** pipeline项目中的精度测试文件为sdk/test目录下的test.py，将test.py移到sdk目录下，执行下面代码，得到pipeline的精度测试结果。 - -``` -python3 test.py -``` - -**步骤4** mxBase项目中，将mxBase目录下main.cpp中main方法的全部代码注释，替换为下面代码后执行（即main函数中仅包含以下代码），得到mxBase的精度测试结果。 - -``` -Test::test_accuracy(); -``` - -## 6 其他问题 -1.本项目的设计为限制输入样例为txt文件，其他文件如图片、音频输入则会报错。 \ No newline at end of file -- Gitee From 5eb59c934835cf448ab3a8aed13fd1ea37f1fe4d Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Fri, 11 Oct 2024 07:05:49 +0000 Subject: [PATCH 02/27] =?UTF-8?q?=E6=9B=B4=E6=96=B0README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/BertTextClassification/README.md | 223 +++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 contrib/BertTextClassification/README.md diff --git a/contrib/BertTextClassification/README.md b/contrib/BertTextClassification/README.md new file mode 100644 index 000000000..13562df60 --- /dev/null +++ b/contrib/BertTextClassification/README.md @@ -0,0 +1,223 @@ +# 文本分类 + +## 1. 介绍 + +### 1.1 简介 +文本分类插件基于 MindXSDK 开发，在晟腾芯片上进行文本分类，将分类结果保存。输入一段新闻，可以判断该新闻属于哪个类别。 +该模型支持5个新闻类别：体育、健康、军事、教育、汽车。 + +### 1.2 支持的产品 + +本项目以昇腾Atlas310卡为主要的硬件平台。 + +### 1.3 支持的版本 + +推荐系统为ubuntu 18.04。 + +表1.1 环境依赖软件和版本： + +| 软件名称 | 版本 | +| -------- | ------ | +| cmake | 3.10.2 | +| mxVision | 2.0.4 | +| python | 3.9.2 | +| CANN | 5.0.4 | + +### 1.4 软件方案介绍 + +基于MindX SDK的文本分类业务流程为：待分类文本通过预处理，将文本根据字典vocab.txt进行编码，组成numpy形式的向量，将向量通过 appsrc 插件输入，然后由模型推理插件mxpi_tensorinfer得到每种类别的得分，再通过后处理插件mxpi_classpostprocessor将模型输出的结果处理，最后得到该文本的类别。本系统的各模块及功能描述如表1.2所示： + + +表1.2 系统方案各子系统功能描述： + +| 序号 | 子系统 | 功能描述 | +| ---- | ------ | ------------ | +| 1 | 文本输入 | 读取输入文本 | +| 2 | 文本编码 | 根据字典对输入文本编码 | +| 3 | 模型推理 | 对文本编码后的张量进行推理 | +| 4 | 后处理 | 从模型推理结果中寻找对应的分类标签 | +| 5 | 保存结果 | 将分类结果保存文件| + +### 1.5 代码目录结构与说明 + +本工程名称为文本分类，工程目录如下图所示： + +``` +. +│ build.sh +│ README.md +│ tree.txt +│ +├─mxBase +│ │ build.sh +│ │ CMakeLists.txt +│ │ main.cpp +│ │ +│ ├─BertClassification +│ │ BertClassification.cpp +│ │ BertClassification.h +│ │ +│ ├─data +│ │ vocab.txt +│ │ +│ ├─model +│ │ bert_text_classification_labels.names +│ │ +│ ├─out +│ │ prediction_label.txt +│ │ +│ └─test +│ Test.cpp +│ Test.h +│ +└─sdk + │ build.sh + │ flowChart.png + │ main.py + │ run.sh + │ tokenizer.py + │ + ├─config + │ bert_text_classification_aipp_tf.cfg + │ bert_text_classification_labels.names + │ + ├─data + │ vocab.txt + │ + ├─model + │ bert_text_classification_aipp_tf.cfg + │ bert_text_classification_labels.names + │ model_conversion.sh + │ + ├─out + │ prediction_label.txt + │ + ├─pipeline + │ BertTextClassification.pipeline + │ + └─test + test.py + test.sh + test_input.py +``` +### 1.5 技术实现流程图 + +![image](sdk/flowChart.png) + + +## 2 设置环境变量 + +确保环境中正确安装mxVision SDK。 + +在编译运行项目前，需要设置环境变量： + +``` +export MX_SDK_HOME=${SDK安装路径}/mxVision +export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:/usr/local/Ascend/driver/lib64:${LD_LIBRARY_PATH} +export PYTHONPATH=${MX_SDK_HOME}/python:${PYTHONPATH} + +export install_path=/usr/local/Ascend/ascend-toolkit/latest +export PATH=/usr/local/python3.9.2/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH +export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH +export ASCEND_OPP_PATH=${install_path}/opp +``` + +- 环境变量介绍 + +``` +MX_SDK_HOME：MindX SDK mxVision的根安装路径，用于包含MindX SDK提供的所有库和头文件。 +LD_LIBRARY_PATH：提供了MindX SDK已开发的插件和相关的库信息。 +install_path：ascend-toolkit的安装路径。 +PATH：添加python的执行路径和atc转换工具的执行路径。 +LD_LIBRARY_PATH：添加ascend-toolkit和MindX SDK提供的库目录路径。 +ASCEND_OPP_PATH：atc转换工具需要的目录。 +``` + +## 3 准备模型 + +**步骤1** 请参考https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/bert_text_classification.pb +下载模型的pb文件，存放到开发环境普通用户下的任意目录，例如：$HOME/models/bert_text_classification。 + +**步骤2** 执行以下命令使用atc命令进行模型转换： + +cd $HOME/models/bert_text_classification + +atc --model=bert_text_classification.pb --framework=3 --input_format="ND" --output=bert_text_classification --input_shape="input_1:1,300;input_2:1,300" --out_nodes=dense_1/Softmax:0 --soc_version=Ascend310 --op_select_implmode="high_precision" + +**步骤3** 执行以下命令将转换好的模型复制到项目中model文件夹中： + +``` +cp ./bert_text_classification.om $HOME/sdk/model/ +cp ./bert_text_classification.om $HOME/mxBase/model/ +``` + +**步骤4** 执行成功后终端输出为： + +``` +ATC start working now, please wait for a moment. +ATC run success, welcome to the next use. +``` + +## 4 编译与运行 + +**步骤1** 从https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip下载测试数据并解压，解压后的sample.txt和test.csv文件放在项目的mxBase/data和sdk/data目录下。 + +**步骤2** 按照第 2 小节 环境依赖 中的步骤设置环境变量。 + +**步骤3** 按照第 3 小节 模型转换 中的步骤获得 om 模型文件。 + +**步骤4** 将本项目代码的文件路径中出现的 ${SDK目录} 替换成自己SDK的存放目录，下面是需要替换的代码。 + +``` +mxBase目录下的CMakeList.txt中的第13行代码： +set(MX_SDK_HOME ${SDK目录}) + +sdk/pipeline目录下BertTextClassification.pipeline文件中的第26行： +"postProcessLibPath": "${SDK目录}/lib/modelpostprocessors/libresnet50postprocess.so" +``` + +**步骤5** pipeline项目运行在sdk目录下执行命令： + +``` +python3 main.py +``` + +命令执行成功后在out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 + +**步骤6** mxBase项目在mxBase目录中，执行以下代码进行编译。 + +``` +mkdir build +cd build +cmake .. +make +``` + +编译完成后，将可执行文件 mxBase_text_classification 移动到mxBase目录下，执行下面代码运行 + +``` +./mxBase_text_classification ./data/sample.txt +``` + +执行成功后在服务器的mxBase/out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 + +## 5 精度测试 + +**步骤1** 已按照上一小节 编译与运行 的步骤将样例运行成功。 + +**步骤2** 下载[数据集](https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip)后解压，将解压后的test.csv文件分别放在sdk/data目录和mxBase/data目录。 + +**步骤3** pipeline项目中的精度测试文件为sdk/test目录下的test.py，将test.py移到sdk目录下，执行下面代码，得到pipeline的精度测试结果。 + +``` +python3 test.py +``` + +**步骤4** mxBase项目中，将mxBase目录下main.cpp中main方法的全部代码注释，替换为下面代码（即main函数中仅包含以下代码），然后重新编译并运行，得到mxBase的精度测试结果。 + +``` +Test::test_accuracy(); +``` + +## 6 其他问题 +1.本项目的设计为限制输入样例为txt文件，其他文件如图片、音频输入则会报错。 \ No newline at end of file -- Gitee From aa170cfafb240d55d5f71ae01ba00fd9275089b3 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Fri, 11 Oct 2024 07:09:15 +0000 Subject: [PATCH 03/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/BertTextClassification/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/BertTextClassification/README.md | 223 ----------------------- 1 file changed, 223 deletions(-) delete mode 100644 contrib/BertTextClassification/README.md diff --git a/contrib/BertTextClassification/README.md b/contrib/BertTextClassification/README.md deleted file mode 100644 index 13562df60..000000000 --- a/contrib/BertTextClassification/README.md +++ /dev/null @@ -1,223 +0,0 @@ -# 文本分类 - -## 1. 介绍 - -### 1.1 简介 -文本分类插件基于 MindXSDK 开发，在晟腾芯片上进行文本分类，将分类结果保存。输入一段新闻，可以判断该新闻属于哪个类别。 -该模型支持5个新闻类别：体育、健康、军事、教育、汽车。 - -### 1.2 支持的产品 - -本项目以昇腾Atlas310卡为主要的硬件平台。 - -### 1.3 支持的版本 - -推荐系统为ubuntu 18.04。 - -表1.1 环境依赖软件和版本： - -| 软件名称 | 版本 | -| -------- | ------ | -| cmake | 3.10.2 | -| mxVision | 2.0.4 | -| python | 3.9.2 | -| CANN | 5.0.4 | - -### 1.4 软件方案介绍 - -基于MindX SDK的文本分类业务流程为：待分类文本通过预处理，将文本根据字典vocab.txt进行编码，组成numpy形式的向量，将向量通过 appsrc 插件输入，然后由模型推理插件mxpi_tensorinfer得到每种类别的得分，再通过后处理插件mxpi_classpostprocessor将模型输出的结果处理，最后得到该文本的类别。本系统的各模块及功能描述如表1.2所示： - - -表1.2 系统方案各子系统功能描述： - -| 序号 | 子系统 | 功能描述 | -| ---- | ------ | ------------ | -| 1 | 文本输入 | 读取输入文本 | -| 2 | 文本编码 | 根据字典对输入文本编码 | -| 3 | 模型推理 | 对文本编码后的张量进行推理 | -| 4 | 后处理 | 从模型推理结果中寻找对应的分类标签 | -| 5 | 保存结果 | 将分类结果保存文件| - -### 1.5 代码目录结构与说明 - -本工程名称为文本分类，工程目录如下图所示： - -``` -. -│ build.sh -│ README.md -│ tree.txt -│ -├─mxBase -│ │ build.sh -│ │ CMakeLists.txt -│ │ main.cpp -│ │ -│ ├─BertClassification -│ │ BertClassification.cpp -│ │ BertClassification.h -│ │ -│ ├─data -│ │ vocab.txt -│ │ -│ ├─model -│ │ bert_text_classification_labels.names -│ │ -│ ├─out -│ │ prediction_label.txt -│ │ -│ └─test -│ Test.cpp -│ Test.h -│ -└─sdk - │ build.sh - │ flowChart.png - │ main.py - │ run.sh - │ tokenizer.py - │ - ├─config - │ bert_text_classification_aipp_tf.cfg - │ bert_text_classification_labels.names - │ - ├─data - │ vocab.txt - │ - ├─model - │ bert_text_classification_aipp_tf.cfg - │ bert_text_classification_labels.names - │ model_conversion.sh - │ - ├─out - │ prediction_label.txt - │ - ├─pipeline - │ BertTextClassification.pipeline - │ - └─test - test.py - test.sh - test_input.py -``` -### 1.5 技术实现流程图 - -![image](sdk/flowChart.png) - - -## 2 设置环境变量 - -确保环境中正确安装mxVision SDK。 - -在编译运行项目前，需要设置环境变量： - -``` -export MX_SDK_HOME=${SDK安装路径}/mxVision -export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:/usr/local/Ascend/driver/lib64:${LD_LIBRARY_PATH} -export PYTHONPATH=${MX_SDK_HOME}/python:${PYTHONPATH} - -export install_path=/usr/local/Ascend/ascend-toolkit/latest -export PATH=/usr/local/python3.9.2/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH -export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH -export ASCEND_OPP_PATH=${install_path}/opp -``` - -- 环境变量介绍 - -``` -MX_SDK_HOME：MindX SDK mxVision的根安装路径，用于包含MindX SDK提供的所有库和头文件。 -LD_LIBRARY_PATH：提供了MindX SDK已开发的插件和相关的库信息。 -install_path：ascend-toolkit的安装路径。 -PATH：添加python的执行路径和atc转换工具的执行路径。 -LD_LIBRARY_PATH：添加ascend-toolkit和MindX SDK提供的库目录路径。 -ASCEND_OPP_PATH：atc转换工具需要的目录。 -``` - -## 3 准备模型 - -**步骤1** 请参考https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/bert_text_classification.pb -下载模型的pb文件，存放到开发环境普通用户下的任意目录，例如：$HOME/models/bert_text_classification。 - -**步骤2** 执行以下命令使用atc命令进行模型转换： - -cd $HOME/models/bert_text_classification - -atc --model=bert_text_classification.pb --framework=3 --input_format="ND" --output=bert_text_classification --input_shape="input_1:1,300;input_2:1,300" --out_nodes=dense_1/Softmax:0 --soc_version=Ascend310 --op_select_implmode="high_precision" - -**步骤3** 执行以下命令将转换好的模型复制到项目中model文件夹中： - -``` -cp ./bert_text_classification.om $HOME/sdk/model/ -cp ./bert_text_classification.om $HOME/mxBase/model/ -``` - -**步骤4** 执行成功后终端输出为： - -``` -ATC start working now, please wait for a moment. -ATC run success, welcome to the next use. -``` - -## 4 编译与运行 - -**步骤1** 从https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip下载测试数据并解压，解压后的sample.txt和test.csv文件放在项目的mxBase/data和sdk/data目录下。 - -**步骤2** 按照第 2 小节 环境依赖 中的步骤设置环境变量。 - -**步骤3** 按照第 3 小节 模型转换 中的步骤获得 om 模型文件。 - -**步骤4** 将本项目代码的文件路径中出现的 ${SDK目录} 替换成自己SDK的存放目录，下面是需要替换的代码。 - -``` -mxBase目录下的CMakeList.txt中的第13行代码： -set(MX_SDK_HOME ${SDK目录}) - -sdk/pipeline目录下BertTextClassification.pipeline文件中的第26行： -"postProcessLibPath": "${SDK目录}/lib/modelpostprocessors/libresnet50postprocess.so" -``` - -**步骤5** pipeline项目运行在sdk目录下执行命令： - -``` -python3 main.py -``` - -命令执行成功后在out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 - -**步骤6** mxBase项目在mxBase目录中，执行以下代码进行编译。 - -``` -mkdir build -cd build -cmake .. -make -``` - -编译完成后，将可执行文件 mxBase_text_classification 移动到mxBase目录下，执行下面代码运行 - -``` -./mxBase_text_classification ./data/sample.txt -``` - -执行成功后在服务器的mxBase/out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 - -## 5 精度测试 - -**步骤1** 已按照上一小节 编译与运行 的步骤将样例运行成功。 - -**步骤2** 下载[数据集](https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip)后解压，将解压后的test.csv文件分别放在sdk/data目录和mxBase/data目录。 - -**步骤3** pipeline项目中的精度测试文件为sdk/test目录下的test.py，将test.py移到sdk目录下，执行下面代码，得到pipeline的精度测试结果。 - -``` -python3 test.py -``` - -**步骤4** mxBase项目中，将mxBase目录下main.cpp中main方法的全部代码注释，替换为下面代码（即main函数中仅包含以下代码），然后重新编译并运行，得到mxBase的精度测试结果。 - -``` -Test::test_accuracy(); -``` - -## 6 其他问题 -1.本项目的设计为限制输入样例为txt文件，其他文件如图片、音频输入则会报错。 \ No newline at end of file -- Gitee From 30494a121a5eefa1b7b0d5a1489fb69643b204a0 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Fri, 11 Oct 2024 07:10:13 +0000 Subject: [PATCH 04/27] =?UTF-8?q?=E6=9B=B4=E6=96=B0README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/BertTextClassification/README.md | 223 +++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 contrib/BertTextClassification/README.md diff --git a/contrib/BertTextClassification/README.md b/contrib/BertTextClassification/README.md new file mode 100644 index 000000000..119c12544 --- /dev/null +++ b/contrib/BertTextClassification/README.md @@ -0,0 +1,223 @@ +# 文本分类 + +## 1. 介绍 + +### 1.1 简介 +文本分类插件基于 MindXSDK 开发，在晟腾芯片上进行文本分类，将分类结果保存。输入一段新闻，可以判断该新闻属于哪个类别。 +该模型支持5个新闻类别：体育、健康、军事、教育、汽车。 + +### 1.2 支持的产品 + +本项目以昇腾Atlas310卡为主要的硬件平台。 + +### 1.3 支持的版本 + +推荐系统为ubuntu 18.04。 + +表1.1 环境依赖软件和版本： + +| 软件名称 | 版本 | +| -------- | ------ | +| cmake | 3.10.2 | +| mxVision | 2.0.4 | +| python | 3.9.2 | +| CANN | 5.0.4 | + +### 1.4 软件方案介绍 + +基于MindX SDK的文本分类业务流程为：待分类文本通过预处理，将文本根据字典vocab.txt进行编码，组成numpy形式的向量，将向量通过 appsrc 插件输入，然后由模型推理插件mxpi_tensorinfer得到每种类别的得分，再通过后处理插件mxpi_classpostprocessor将模型输出的结果处理，最后得到该文本的类别。本系统的各模块及功能描述如表1.2所示： + + +表1.2 系统方案各子系统功能描述： + +| 序号 | 子系统 | 功能描述 | +| ---- | ------ | ------------ | +| 1 | 文本输入 | 读取输入文本 | +| 2 | 文本编码 | 根据字典对输入文本编码 | +| 3 | 模型推理 | 对文本编码后的张量进行推理 | +| 4 | 后处理 | 从模型推理结果中寻找对应的分类标签 | +| 5 | 保存结果 | 将分类结果保存文件| + +### 1.5 代码目录结构与说明 + +本工程名称为文本分类，工程目录如下图所示： + +``` +. +│ build.sh +│ README.md +│ tree.txt +│ +├─mxBase +│ │ build.sh +│ │ CMakeLists.txt +│ │ main.cpp +│ │ +│ ├─BertClassification +│ │ BertClassification.cpp +│ │ BertClassification.h +│ │ +│ ├─data +│ │ vocab.txt +│ │ +│ ├─model +│ │ bert_text_classification_labels.names +│ │ +│ ├─out +│ │ prediction_label.txt +│ │ +│ └─test +│ Test.cpp +│ Test.h +│ +└─sdk + │ build.sh + │ flowChart.png + │ main.py + │ run.sh + │ tokenizer.py + │ + ├─config + │ bert_text_classification_aipp_tf.cfg + │ bert_text_classification_labels.names + │ + ├─data + │ vocab.txt + │ + ├─model + │ bert_text_classification_aipp_tf.cfg + │ bert_text_classification_labels.names + │ model_conversion.sh + │ + ├─out + │ prediction_label.txt + │ + ├─pipeline + │ BertTextClassification.pipeline + │ + └─test + test.py + test.sh + test_input.py +``` +### 1.6 技术实现流程图 + +![image](sdk/flowChart.png) + + +## 2 设置环境变量 + +确保环境中正确安装mxVision SDK。 + +在编译运行项目前，需要设置环境变量： + +``` +export MX_SDK_HOME=${SDK安装路径}/mxVision +export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:/usr/local/Ascend/driver/lib64:${LD_LIBRARY_PATH} +export PYTHONPATH=${MX_SDK_HOME}/python:${PYTHONPATH} + +export install_path=/usr/local/Ascend/ascend-toolkit/latest +export PATH=/usr/local/python3.9.2/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH +export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH +export ASCEND_OPP_PATH=${install_path}/opp +``` + +- 环境变量介绍 + +``` +MX_SDK_HOME：MindX SDK mxVision的根安装路径，用于包含MindX SDK提供的所有库和头文件。 +LD_LIBRARY_PATH：提供了MindX SDK已开发的插件和相关的库信息。 +install_path：ascend-toolkit的安装路径。 +PATH：添加python的执行路径和atc转换工具的执行路径。 +LD_LIBRARY_PATH：添加ascend-toolkit和MindX SDK提供的库目录路径。 +ASCEND_OPP_PATH：atc转换工具需要的目录。 +``` + +## 3 准备模型 + +**步骤1** 请参考https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/bert_text_classification.pb +下载模型的pb文件，存放到开发环境普通用户下的任意目录，例如：$HOME/models/bert_text_classification。 + +**步骤2** 执行以下命令使用atc命令进行模型转换： + +cd $HOME/models/bert_text_classification + +atc --model=bert_text_classification.pb --framework=3 --input_format="ND" --output=bert_text_classification --input_shape="input_1:1,300;input_2:1,300" --out_nodes=dense_1/Softmax:0 --soc_version=Ascend310 --op_select_implmode="high_precision" + +**步骤3** 执行以下命令将转换好的模型复制到项目中model文件夹中： + +``` +cp ./bert_text_classification.om $HOME/sdk/model/ +cp ./bert_text_classification.om $HOME/mxBase/model/ +``` + +**步骤4** 执行成功后终端输出为： + +``` +ATC start working now, please wait for a moment. +ATC run success, welcome to the next use. +``` + +## 4 编译与运行 + +**步骤1** 从https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip下载测试数据并解压，解压后的sample.txt和test.csv文件放在项目的mxBase/data和sdk/data目录下。 + +**步骤2** 按照第 2 小节 环境依赖 中的步骤设置环境变量。 + +**步骤3** 按照第 3 小节 模型转换 中的步骤获得 om 模型文件。 + +**步骤4** 将本项目代码的文件路径中出现的 ${SDK目录} 替换成自己SDK的存放目录，下面是需要替换的代码。 + +``` +mxBase目录下的CMakeList.txt中的第13行代码： +set(MX_SDK_HOME ${SDK目录}) + +sdk/pipeline目录下BertTextClassification.pipeline文件中的第26行： +"postProcessLibPath": "${SDK目录}/lib/modelpostprocessors/libresnet50postprocess.so" +``` + +**步骤5** pipeline项目运行在sdk目录下执行命令： + +``` +python3 main.py +``` + +命令执行成功后在out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 + +**步骤6** mxBase项目在mxBase目录中，执行以下代码进行编译。 + +``` +mkdir build +cd build +cmake .. +make +``` + +编译完成后，将可执行文件 mxBase_text_classification 移动到mxBase目录下，执行下面代码运行 + +``` +./mxBase_text_classification ./data/sample.txt +``` + +执行成功后在服务器的mxBase/out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 + +## 5 精度测试 + +**步骤1** 已按照上一小节 编译与运行 的步骤将样例运行成功。 + +**步骤2** 下载[数据集](https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip)后解压，将解压后的test.csv文件分别放在sdk/data目录和mxBase/data目录。 + +**步骤3** pipeline项目中的精度测试文件为sdk/test目录下的test.py，将test.py移到sdk目录下，执行下面代码，得到pipeline的精度测试结果。 + +``` +python3 test.py +``` + +**步骤4** mxBase项目中，将mxBase目录下main.cpp中main方法的全部代码注释，替换为下面代码（即main函数中仅包含以下代码），参考第4小节 编译与运行 中的步骤4重新编译并运行，得到mxBase的精度测试结果。 + +``` +Test::test_accuracy(); +``` + +## 6 常见问题 +1.本项目的设计为限制输入样例为txt文件，其他文件如图片、音频输入则会报错。 \ No newline at end of file -- Gitee From 4d96525ce6ff01f84c12788298128d46a0233f43 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:13:23 +0000 Subject: [PATCH 05/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/CMakeLists.txt?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/CMakeLists.txt | 37 --------------------- 1 file changed, 37 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/CMakeLists.txt diff --git a/contrib/EdgeDetectionPicture/CMakeLists.txt b/contrib/EdgeDetectionPicture/CMakeLists.txt deleted file mode 100644 index 3928c17d7..000000000 --- a/contrib/EdgeDetectionPicture/CMakeLists.txt +++ /dev/null @@ -1,37 +0,0 @@ -cmake_minimum_required(VERSION 3.10) -project(edge_detection_picture) - -include_directories(./rcfPostProcess) -include_directories(./rcfDetection) -file(GLOB_RECURSE RCF_POSTPROCESS ${PROJECT_SOURCE_DIR}/rcfPostProcess/*cpp) -file(GLOB_RECURSE RCF_DETECTION ${PROJECT_SOURCE_DIR}/rcfDetection/*cpp) -set(TARGET edge_detection_picture) -add_compile_options(-std=c++11 -fPIE -fstack-protector-all -fPIC -Wl,-z,relro,-z,now,-z,noexecstack -s -pie -Wall) -add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) - -set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") - -include_directories( - ${MX_SDK_HOME}/include - ${MX_SDK_HOME}/opensource/include - ${MX_SDK_HOME}/opensource/include/opencv4 -) - -link_directories( - ${MX_SDK_HOME}/lib - ${MX_SDK_HOME}/opensource/lib - ${MX_SDK_HOME}/lib/modelpostprocessors - ${MX_SDK_HOME}/include/MxBase/postprocess/include - /usr/local/Ascend/ascend-toolkit/latest/acllib/lib64 - /usr/local/Ascend/driver/lib64/ -) - -add_executable(edge_detection_picture main.cpp ${RCF_DETECTION} ${RCF_POSTPROCESS}) -target_link_libraries(edge_detection_picture - glog - mxbase - opencv_world - boost_system - boost_filesystem - ) -install(TARGETS ${TARGET} RUNTIME DESTINATION ${PROJECT_SOURCE_DIR}/) -- Gitee From 061a83867b246235cdfa347191776a192943782a Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:13:50 +0000 Subject: [PATCH 06/27] =?UTF-8?q?=E6=9B=B4=E6=96=B0CMake=E4=BD=BF=E5=85=B6?= =?UTF-8?q?=E9=80=82=E9=85=8D=E7=89=88=E6=9C=AC?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/EdgeDetectionPicture/CMakeLists.txt | 43 +++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 contrib/EdgeDetectionPicture/CMakeLists.txt diff --git a/contrib/EdgeDetectionPicture/CMakeLists.txt b/contrib/EdgeDetectionPicture/CMakeLists.txt new file mode 100644 index 000000000..c038f4269 --- /dev/null +++ b/contrib/EdgeDetectionPicture/CMakeLists.txt @@ -0,0 +1,43 @@ +cmake_minimum_required(VERSION 3.10) +project(edge_detection_picture) + +include_directories(./rcfPostProcess) +include_directories(./rcfDetection) +file(GLOB_RECURSE RCF_POSTPROCESS ${PROJECT_SOURCE_DIR}/rcfPostProcess/*cpp) +file(GLOB_RECURSE RCF_DETECTION ${PROJECT_SOURCE_DIR}/rcfDetection/*cpp) +set(TARGET edge_detection_picture) +add_compile_options(-std=c++11 -fPIE -fstack-protector-all -fPIC -Wl,-z,relro,-z,now,-z,noexecstack -s -pie -Wall) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) + +set(MX_SDK_HOME /root/SDK/mxVision) + +set(cpprest_DIR ${MX_SDK_HOME}/opensource/lib/libcpprest.so) +if(EXISTS ${cpprest_DIR}) + target_link_libraries(edge_detection_picture cpprest) + add_definitions(_DMX_VERSION_5) +endif() + +include_directories( + ${MX_SDK_HOME}/include + ${MX_SDK_HOME}/opensource/include + ${MX_SDK_HOME}/opensource/include/opencv4 +) + +link_directories( + ${MX_SDK_HOME}/lib + ${MX_SDK_HOME}/opensource/lib + ${MX_SDK_HOME}/lib/modelpostprocessors + ${MX_SDK_HOME}/include/MxBase/postprocess/include + /usr/local/Ascend/ascend-toolkit/latest/acllib/lib64 + /usr/local/Ascend/driver/lib64/ +) + +add_executable(edge_detection_picture main.cpp ${RCF_DETECTION} ${RCF_POSTPROCESS}) +target_link_libraries(edge_detection_picture + glog + mxbase + opencv_world + boost_system + boost_filesystem + ) +install(TARGETS ${TARGET} RUNTIME DESTINATION ${PROJECT_SOURCE_DIR}/) -- Gitee From fb9df239a505356210a7f2c62c34abb014b5757e Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:14:00 +0000 Subject: [PATCH 07/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/rcfDetection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../rcfDetection/RcfDetection.cpp | 281 ------------------ .../rcfDetection/RcfDetection.h | 61 ---- 2 files changed, 342 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp delete mode 100644 contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h diff --git a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp deleted file mode 100644 index a8a84460c..000000000 --- a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp +++ /dev/null @@ -1,281 +0,0 @@ -/** - * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#include "RcfDetection.h" -#include "MxBase/DeviceManager/DeviceManager.h" -#include "MxBase/Log/Log.h" -#include "../rcfPostProcess/RcfPostProcess.h" -#include -#include -#include "opencv2/opencv.hpp" - -namespace { - const uint32_t YUV_BYTE_NU = 3; - const uint32_t YUV_BYTE_DE = 2; - const uint32_t VPC_H_ALIGN = 2; - const uint32_t channel = 3; - const double ALPHA = 255.0 / 255; -} -void RcfDetection::SetRcfPostProcessConfig(const InitParam &initParam, - std::map &config) -{ - MxBase::ConfigData configData; - const std::string checkTensor = initParam.checkTensor ? "true" : "false"; - configData.SetJsonValue("OUTSIZE_NUM", std::to_string(initParam.outSizeNum)); - configData.SetJsonValue("OUTSIZE", initParam.outSize); - configData.SetJsonValue("RCF_TYPE", std::to_string(initParam.rcfType)); - configData.SetJsonValue("MODEL_TYPE", std::to_string(initParam.modelType)); - configData.SetJsonValue("INPUT_TYPE", std::to_string(initParam.inputType)); - configData.SetJsonValue("CHECK_MODEL", checkTensor); - auto jsonStr = configData.GetCfgJson().serialize(); - config["postProcessConfigContent"] = jsonStr; -} - -APP_ERROR RcfDetection::Init(const InitParam &initParam) -{ - deviceId_ = initParam.deviceId; - APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices(); - if (ret != APP_ERR_OK) { - LogError << "Init devices failed, ret=" << ret << "."; - return ret; - } - ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId); - if (ret != APP_ERR_OK) { - LogError << "Set context failed, ret=" << ret << "."; - return ret; - } - dvppWrapper_ = std::make_shared(); - ret = dvppWrapper_->Init(); - if (ret != APP_ERR_OK) { - LogError << "DvppWrapper init failed, ret=" << ret << "."; - return ret; - } - model_ = std::make_shared(); - ret = model_->Init(initParam.modelPath, modelDesc_); - if (ret != APP_ERR_OK) { - LogError << "ModelInferenceProcessor init failed, ret=" << ret << "."; - return ret; - } - - std::map config; - SetRcfPostProcessConfig(initParam, config); - post_ = std::make_shared(); - ret = post_->Init(config); - if (ret != APP_ERR_OK) { - LogError << "RcfPostProcess init failed, ret=" << ret << "."; - return ret; - } - if (ret != APP_ERR_OK) { - LogError << "Failed to load labels, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::DeInit() -{ - dvppWrapper_->DeInit(); - model_->DeInit(); - post_->DeInit(); - MxBase::DeviceManager::GetInstance()->DestroyDevices(); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::ReadImage(const std::string &imgPath, - MxBase::TensorBase &tensor) -{ - MxBase::DvppDataInfo output = {}; - APP_ERROR ret = dvppWrapper_->DvppJpegDecode(imgPath, output); - if (ret != APP_ERR_OK) { - LogError << "DvppWrapper DvppJpegDecode failed, ret=" << ret << "."; - return ret; - } - MxBase::MemoryData memoryData((void*)output.data, output.dataSize, - MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - if (output.heightStride % VPC_H_ALIGN != 0) { - LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; - MxBase::MemoryHelper::MxbsFree(memoryData); - return APP_ERR_COMM_INVALID_PARAM; - } - dvppHeightStride = output.heightStride; - dvppWidthStride = output.widthStride; - std::vector shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; - tensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); - return APP_ERR_OK; -} -APP_ERROR RcfDetection::Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, - uint32_t resizeHeight, uint32_t resizeWidth) -{ - auto shape = inputTensor.GetShape(); - MxBase::DvppDataInfo input = {}; - input.height = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; - input.width = shape[1]; - input.heightStride = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; - input.widthStride = shape[1]; - input.dataSize = inputTensor.GetByteSize(); - input.data = (uint8_t*)inputTensor.GetBuffer(); - MxBase::ResizeConfig resize = {}; - resize.height = resizeHeight; - resize.width = resizeWidth; - MxBase::DvppDataInfo output = {}; - APP_ERROR ret = dvppWrapper_->VpcResize(input, output, resize); - if (ret != APP_ERR_OK) { - LogError << "VpcResize failed, ret=" << ret << "."; - return ret; - } - MxBase::MemoryData memoryData((void*)output.data, output.dataSize, - MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - if (output.heightStride % VPC_H_ALIGN != 0) { - LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; - MxBase::MemoryHelper::MxbsFree(memoryData); - return APP_ERR_COMM_INVALID_PARAM; - } - shape = {1, channel, output.heightStride, output.widthStride}; - outputTensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::Inference(const std::vector &inputs, - std::vector &outputs) -{ - std::vector output={}; - auto dtypes = model_->GetOutputDataType(); - for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) { - std::vector shape = {}; - for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) { - shape.push_back((uint32_t)modelDesc_.outputTensors[i].tensorDims[j]); - } - MxBase::TensorBase tensor(shape, dtypes[i], MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor); - if (ret != APP_ERR_OK) { - LogError << "TensorBaseMalloc failed, ret=" << ret << "."; - return ret; - } - outputs.push_back(tensor); - } - MxBase::DynamicInfo dynamicInfo = {}; - dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH; - auto startTime = std::chrono::high_resolution_clock::now(); - APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo); - auto endTime = std::chrono::high_resolution_clock::now(); - double costMs = std::chrono::duration(endTime - startTime).count(); - LogInfo<< "costMs:"<< costMs; - if (ret != APP_ERR_OK) { - LogError << "ModelInference failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::PostProcess(const MxBase::TensorBase &tensor, - const std::vector &outputs, - std::vector &postProcessOutput) -{ - auto shape = tensor.GetShape(); - MxBase::ResizedImageInfo imgInfo; - imgInfo.widthOriginal = shape[1]; - imgInfo.heightOriginal = shape[0] * YUV_BYTE_DE; - uint32_t widthResize = 512; - uint32_t heightResize = 512; - imgInfo.widthResize = widthResize; - imgInfo.heightResize = heightResize; - imgInfo.resizeType = MxBase::RESIZER_STRETCHING; - std::vector imageInfoVec = {}; - imageInfoVec.push_back(imgInfo); - APP_ERROR ret = post_->Process(outputs, postProcessOutput); - if (ret != APP_ERR_OK) { - LogError << "Process failed, ret=" << ret << "."; - return ret; - } - ret = post_->DeInit(); - if (ret != APP_ERR_OK) { - LogError << "RcfDetection DeInit failed"; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath) -{ - auto shape = inferTensor.GetShape(); - int dim2 = 2; - int dim3 = 3; - uint32_t height = shape[dim2]; - uint32_t width = shape[dim3]; - cv::Mat imgBgr = cv::imread(imgPath); - std::string fileName = imgPath.substr(imgPath.find_last_of("/") + 1); - uint32_t imageWidth = imgBgr.cols; - uint32_t imageHeight = imgBgr.rows; - cv::Mat modelOutput = cv::Mat(height, width, CV_32FC1, inferTensor.GetBuffer()); - cv::Mat grayMat; - cv::Mat resizedMat; - int crop = 5; - cv::Rect myROI(0, 0, imageWidth - crop, imageHeight); - resize(modelOutput, resizedMat, cv::Size(dvppWidthStride, dvppHeightStride), 0, 0, cv::INTER_LINEAR); - resizedMat.convertTo(grayMat, CV_8UC1, ALPHA); - cv::Mat croppedImage = grayMat(myROI); - resize(croppedImage, croppedImage, cv::Size(imageWidth, imageHeight), 0, 0, cv::INTER_LINEAR); - - std::string resultPathName = "result"; - if (access(resultPathName.c_str(), 0) != 0) { - int ret = mkdir(resultPathName.c_str(), S_IRUSR | S_IWUSR | S_IXUSR); - if (ret != 0) { - LogError << "Failed to create result directory: " << resultPathName << ", ret = " << ret; - return APP_ERR_COMM_FAILURE; - } - } - - cv::imwrite("./result/"+fileName, croppedImage); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::Process(const std::string &imgPath) -{ - MxBase::TensorBase inTensor; - APP_ERROR ret = ReadImage(imgPath, inTensor); - if (ret != APP_ERR_OK) { - LogError << "ReadImage failed, ret=" << ret << "."; - return ret; - } - MxBase::TensorBase outTensor; - uint32_t resizeHeight = 512; - uint32_t resizeWidth = 512; - ret = Resize(inTensor, outTensor, resizeHeight, resizeWidth); - if (ret != APP_ERR_OK) { - LogError << "Resize failed, ret=" << ret << "."; - return ret; - } - std::vector inputs = {}; - std::vector outputs = {}; - auto shape = outTensor.GetShape(); - inputs.push_back(outTensor); - ret = Inference(inputs, outputs); - if (ret != APP_ERR_OK) { - LogError << "Inference failed, ret=" << ret << "."; - return ret; - } - std::vector postProcessOutput={}; - ret = PostProcess(inTensor, outputs, postProcessOutput); - if (ret != APP_ERR_OK) { - LogError << "PostProcess failed, ret=" << ret << "."; - return ret; - } - ret = WriteResult(postProcessOutput[0], imgPath); - if (ret != APP_ERR_OK) { - LogError << "Save result failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} diff --git a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h deleted file mode 100644 index 7c0ae6da4..000000000 --- a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h +++ /dev/null @@ -1,61 +0,0 @@ -/* - * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#ifndef MXBASE_RCFDETECTION_H -#define MXBASE_RCFDETECTION_H - -#include -#include -#include "MxBase/DvppWrapper/DvppWrapper.h" -#include "MxBase/ModelInfer/ModelInferenceProcessor.h" -#include "MxBase/Tensor/TensorContext/TensorContext.h" - -struct InitParam { - uint32_t deviceId = 0; - bool checkTensor = true; - std::string modelPath = ""; - uint32_t outSizeNum = 0; - std::string outSize = ""; - uint32_t rcfType = 0; - uint32_t modelType = 0; - uint32_t inputType = 0; -}; - -class RcfDetection { -public: - APP_ERROR Init(const InitParam &initParam); - APP_ERROR DeInit(); - APP_ERROR Inference(const std::vector &inputs, std::vector &outputs); - APP_ERROR PostProcess(const MxBase::TensorBase &tensor, const std::vector &outputs, - std::vector &postProcessOutput); - APP_ERROR Process(const std::string &imgPath); -protected: - APP_ERROR ReadImage(const std::string &imgPath, MxBase::TensorBase &tensor); - APP_ERROR Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, - uint32_t resizeHeight, uint32_t resizeWidth); - APP_ERROR WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath); - void SetRcfPostProcessConfig(const InitParam &initParam, std::map &config); - -private: - std::shared_ptr dvppWrapper_; - std::shared_ptr model_; - std::shared_ptr post_; - MxBase::ModelDesc modelDesc_ = {}; - uint32_t deviceId_ = 0; - int dvppHeightStride = 0; - int dvppWidthStride = 0; -}; -#endif -- Gitee From 5a5c1e129e621eb7dd649e47bf5a9a9004c00f3c Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:14:14 +0000 Subject: [PATCH 08/27] =?UTF-8?q?=E6=96=B0=E5=BB=BA=20rcfDetection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/rcfDetection/.keep | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 contrib/EdgeDetectionPicture/rcfDetection/.keep diff --git a/contrib/EdgeDetectionPicture/rcfDetection/.keep b/contrib/EdgeDetectionPicture/rcfDetection/.keep new file mode 100644 index 000000000..e69de29bb -- Gitee From 9eaf6d186dd3a44d9db2e9cac335c85439f1a6b2 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:15:03 +0000 Subject: [PATCH 09/27] =?UTF-8?q?=E6=9B=B4=E6=96=B0RcfDetection=E4=BD=BF?= =?UTF-8?q?=E5=85=B6=E9=80=82=E9=85=8D=E7=89=88=E6=9C=AC?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/EdgeDetectionPicture/RcfDetection.cpp | 286 ++++++++++++++++++ contrib/EdgeDetectionPicture/RcfDetection.h | 61 ++++ 2 files changed, 347 insertions(+) create mode 100644 contrib/EdgeDetectionPicture/RcfDetection.cpp create mode 100644 contrib/EdgeDetectionPicture/RcfDetection.h diff --git a/contrib/EdgeDetectionPicture/RcfDetection.cpp b/contrib/EdgeDetectionPicture/RcfDetection.cpp new file mode 100644 index 000000000..ab1796d72 --- /dev/null +++ b/contrib/EdgeDetectionPicture/RcfDetection.cpp @@ -0,0 +1,286 @@ +/** + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "RcfDetection.h" +#include "MxBase/DeviceManager/DeviceManager.h" +#include "MxBase/Log/Log.h" +#include "../rcfPostProcess/RcfPostProcess.h" +#include +#include +#include "opencv2/opencv.hpp" + +namespace { + const uint32_t YUV_BYTE_NU = 3; + const uint32_t YUV_BYTE_DE = 2; + const uint32_t VPC_H_ALIGN = 2; + const uint32_t channel = 3; + const double ALPHA = 255.0 / 255; +} +void RcfDetection::SetRcfPostProcessConfig(const InitParam &initParam, + std::map &config) +{ + MxBase::ConfigData configData; + const std::string checkTensor = initParam.checkTensor ? "true" : "false"; + configData.SetJsonValue("OUTSIZE_NUM", std::to_string(initParam.outSizeNum)); + configData.SetJsonValue("OUTSIZE", initParam.outSize); + configData.SetJsonValue("RCF_TYPE", std::to_string(initParam.rcfType)); + configData.SetJsonValue("MODEL_TYPE", std::to_string(initParam.modelType)); + configData.SetJsonValue("INPUT_TYPE", std::to_string(initParam.inputType)); + configData.SetJsonValue("CHECK_MODEL", checkTensor); + #ifdef MX_VERSION_5 + auto jsonStr = configData.GetCfgJson().serialize(); + config["postProcessConfigContent"] = jsonStr; + #else + auto jsonStr = configData.GetCfgJson(); + config["postProcessConfigContent"] = jsonStr; + #endif +} + +APP_ERROR RcfDetection::Init(const InitParam &initParam) +{ + deviceId_ = initParam.deviceId; + APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices(); + if (ret != APP_ERR_OK) { + LogError << "Init devices failed, ret=" << ret << "."; + return ret; + } + ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId); + if (ret != APP_ERR_OK) { + LogError << "Set context failed, ret=" << ret << "."; + return ret; + } + dvppWrapper_ = std::make_shared(); + ret = dvppWrapper_->Init(); + if (ret != APP_ERR_OK) { + LogError << "DvppWrapper init failed, ret=" << ret << "."; + return ret; + } + model_ = std::make_shared(); + ret = model_->Init(initParam.modelPath, modelDesc_); + if (ret != APP_ERR_OK) { + LogError << "ModelInferenceProcessor init failed, ret=" << ret << "."; + return ret; + } + + std::map config; + SetRcfPostProcessConfig(initParam, config); + post_ = std::make_shared(); + ret = post_->Init(config); + if (ret != APP_ERR_OK) { + LogError << "RcfPostProcess init failed, ret=" << ret << "."; + return ret; + } + if (ret != APP_ERR_OK) { + LogError << "Failed to load labels, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::DeInit() +{ + dvppWrapper_->DeInit(); + model_->DeInit(); + post_->DeInit(); + MxBase::DeviceManager::GetInstance()->DestroyDevices(); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::ReadImage(const std::string &imgPath, + MxBase::TensorBase &tensor) +{ + MxBase::DvppDataInfo output = {}; + APP_ERROR ret = dvppWrapper_->DvppJpegDecode(imgPath, output); + if (ret != APP_ERR_OK) { + LogError << "DvppWrapper DvppJpegDecode failed, ret=" << ret << "."; + return ret; + } + MxBase::MemoryData memoryData((void*)output.data, output.dataSize, + MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + if (output.heightStride % VPC_H_ALIGN != 0) { + LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; + MxBase::MemoryHelper::MxbsFree(memoryData); + return APP_ERR_COMM_INVALID_PARAM; + } + dvppHeightStride = output.heightStride; + dvppWidthStride = output.widthStride; + std::vector shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; + tensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); + return APP_ERR_OK; +} +APP_ERROR RcfDetection::Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, + uint32_t resizeHeight, uint32_t resizeWidth) +{ + auto shape = inputTensor.GetShape(); + MxBase::DvppDataInfo input = {}; + input.height = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; + input.width = shape[1]; + input.heightStride = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; + input.widthStride = shape[1]; + input.dataSize = inputTensor.GetByteSize(); + input.data = (uint8_t*)inputTensor.GetBuffer(); + MxBase::ResizeConfig resize = {}; + resize.height = resizeHeight; + resize.width = resizeWidth; + MxBase::DvppDataInfo output = {}; + APP_ERROR ret = dvppWrapper_->VpcResize(input, output, resize); + if (ret != APP_ERR_OK) { + LogError << "VpcResize failed, ret=" << ret << "."; + return ret; + } + MxBase::MemoryData memoryData((void*)output.data, output.dataSize, + MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + if (output.heightStride % VPC_H_ALIGN != 0) { + LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; + MxBase::MemoryHelper::MxbsFree(memoryData); + return APP_ERR_COMM_INVALID_PARAM; + } + shape = {1, channel, output.heightStride, output.widthStride}; + outputTensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::Inference(const std::vector &inputs, + std::vector &outputs) +{ + std::vector output={}; + auto dtypes = model_->GetOutputDataType(); + for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) { + std::vector shape = {}; + for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) { + shape.push_back((uint32_t)modelDesc_.outputTensors[i].tensorDims[j]); + } + MxBase::TensorBase tensor(shape, dtypes[i], MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor); + if (ret != APP_ERR_OK) { + LogError << "TensorBaseMalloc failed, ret=" << ret << "."; + return ret; + } + outputs.push_back(tensor); + } + MxBase::DynamicInfo dynamicInfo = {}; + dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH; + auto startTime = std::chrono::high_resolution_clock::now(); + APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo); + auto endTime = std::chrono::high_resolution_clock::now(); + double costMs = std::chrono::duration(endTime - startTime).count(); + LogInfo<< "costMs:"<< costMs; + if (ret != APP_ERR_OK) { + LogError << "ModelInference failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::PostProcess(const MxBase::TensorBase &tensor, + const std::vector &outputs, + std::vector &postProcessOutput) +{ + auto shape = tensor.GetShape(); + MxBase::ResizedImageInfo imgInfo; + imgInfo.widthOriginal = shape[1]; + imgInfo.heightOriginal = shape[0] * YUV_BYTE_DE; + uint32_t widthResize = 512; + uint32_t heightResize = 512; + imgInfo.widthResize = widthResize; + imgInfo.heightResize = heightResize; + imgInfo.resizeType = MxBase::RESIZER_STRETCHING; + std::vector imageInfoVec = {}; + imageInfoVec.push_back(imgInfo); + APP_ERROR ret = post_->Process(outputs, postProcessOutput); + if (ret != APP_ERR_OK) { + LogError << "Process failed, ret=" << ret << "."; + return ret; + } + ret = post_->DeInit(); + if (ret != APP_ERR_OK) { + LogError << "RcfDetection DeInit failed"; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath) +{ + auto shape = inferTensor.GetShape(); + int dim2 = 2; + int dim3 = 3; + uint32_t height = shape[dim2]; + uint32_t width = shape[dim3]; + cv::Mat imgBgr = cv::imread(imgPath); + std::string fileName = imgPath.substr(imgPath.find_last_of("/") + 1); + uint32_t imageWidth = imgBgr.cols; + uint32_t imageHeight = imgBgr.rows; + cv::Mat modelOutput = cv::Mat(height, width, CV_32FC1, inferTensor.GetBuffer()); + cv::Mat grayMat; + cv::Mat resizedMat; + int crop = 5; + cv::Rect myROI(0, 0, imageWidth - crop, imageHeight); + resize(modelOutput, resizedMat, cv::Size(dvppWidthStride, dvppHeightStride), 0, 0, cv::INTER_LINEAR); + resizedMat.convertTo(grayMat, CV_8UC1, ALPHA); + cv::Mat croppedImage = grayMat(myROI); + resize(croppedImage, croppedImage, cv::Size(imageWidth, imageHeight), 0, 0, cv::INTER_LINEAR); + + std::string resultPathName = "result"; + if (access(resultPathName.c_str(), 0) != 0) { + int ret = mkdir(resultPathName.c_str(), S_IRUSR | S_IWUSR | S_IXUSR); + if (ret != 0) { + LogError << "Failed to create result directory: " << resultPathName << ", ret = " << ret; + return APP_ERR_COMM_FAILURE; + } + } + + cv::imwrite("./result/"+fileName, croppedImage); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::Process(const std::string &imgPath) +{ + MxBase::TensorBase inTensor; + APP_ERROR ret = ReadImage(imgPath, inTensor); + if (ret != APP_ERR_OK) { + LogError << "ReadImage failed, ret=" << ret << "."; + return ret; + } + MxBase::TensorBase outTensor; + uint32_t resizeHeight = 512; + uint32_t resizeWidth = 512; + ret = Resize(inTensor, outTensor, resizeHeight, resizeWidth); + if (ret != APP_ERR_OK) { + LogError << "Resize failed, ret=" << ret << "."; + return ret; + } + std::vector inputs = {}; + std::vector outputs = {}; + auto shape = outTensor.GetShape(); + inputs.push_back(outTensor); + ret = Inference(inputs, outputs); + if (ret != APP_ERR_OK) { + LogError << "Inference failed, ret=" << ret << "."; + return ret; + } + std::vector postProcessOutput={}; + ret = PostProcess(inTensor, outputs, postProcessOutput); + if (ret != APP_ERR_OK) { + LogError << "PostProcess failed, ret=" << ret << "."; + return ret; + } + ret = WriteResult(postProcessOutput[0], imgPath); + if (ret != APP_ERR_OK) { + LogError << "Save result failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} diff --git a/contrib/EdgeDetectionPicture/RcfDetection.h b/contrib/EdgeDetectionPicture/RcfDetection.h new file mode 100644 index 000000000..7c0ae6da4 --- /dev/null +++ b/contrib/EdgeDetectionPicture/RcfDetection.h @@ -0,0 +1,61 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef MXBASE_RCFDETECTION_H +#define MXBASE_RCFDETECTION_H + +#include +#include +#include "MxBase/DvppWrapper/DvppWrapper.h" +#include "MxBase/ModelInfer/ModelInferenceProcessor.h" +#include "MxBase/Tensor/TensorContext/TensorContext.h" + +struct InitParam { + uint32_t deviceId = 0; + bool checkTensor = true; + std::string modelPath = ""; + uint32_t outSizeNum = 0; + std::string outSize = ""; + uint32_t rcfType = 0; + uint32_t modelType = 0; + uint32_t inputType = 0; +}; + +class RcfDetection { +public: + APP_ERROR Init(const InitParam &initParam); + APP_ERROR DeInit(); + APP_ERROR Inference(const std::vector &inputs, std::vector &outputs); + APP_ERROR PostProcess(const MxBase::TensorBase &tensor, const std::vector &outputs, + std::vector &postProcessOutput); + APP_ERROR Process(const std::string &imgPath); +protected: + APP_ERROR ReadImage(const std::string &imgPath, MxBase::TensorBase &tensor); + APP_ERROR Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, + uint32_t resizeHeight, uint32_t resizeWidth); + APP_ERROR WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath); + void SetRcfPostProcessConfig(const InitParam &initParam, std::map &config); + +private: + std::shared_ptr dvppWrapper_; + std::shared_ptr model_; + std::shared_ptr post_; + MxBase::ModelDesc modelDesc_ = {}; + uint32_t deviceId_ = 0; + int dvppHeightStride = 0; + int dvppWidthStride = 0; +}; +#endif -- Gitee From d3e96ed7244023009439f4c714e0265842a3200f Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:15:29 +0000 Subject: [PATCH 10/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/rcfDetection/.keep?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/rcfDetection/.keep | 0 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/rcfDetection/.keep diff --git a/contrib/EdgeDetectionPicture/rcfDetection/.keep b/contrib/EdgeDetectionPicture/rcfDetection/.keep deleted file mode 100644 index e69de29bb..000000000 -- Gitee From 2c3f8c16725c766066bccc8207571a1c5971ad70 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:16:18 +0000 Subject: [PATCH 11/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/README.md | 150 ------------------------- 1 file changed, 150 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/README.md diff --git a/contrib/EdgeDetectionPicture/README.md b/contrib/EdgeDetectionPicture/README.md deleted file mode 100644 index e00f69a20..000000000 --- a/contrib/EdgeDetectionPicture/README.md +++ /dev/null @@ -1,150 +0,0 @@ - -# RCF模型边缘检测 - -## 1 介绍 -本开发样例是基于mxBase开发的端到端推理的C++应用程序，可在昇腾芯片上进行 图像边缘提取，并把可视化结果保存到本地。 -其中包含Rcf模型的后处理模块开发。 主要处理流程为： Init > ReadImage >Resize > Inference >PostProcess >DeInit - -#### 1.1 支持的产品 -昇腾310(推理) - -#### 1.2 支持的版本 -本样例配套的CANN版本为7.0.0，MindX SDK版本为5.0.0 -MindX SDK安装前准备可参考《用户指南》，[安装教程](https://gitee.com/ascend/mindxsdk-referenceapps/blob/master/docs/quickStart/1-1%E5%AE%89%E8%A3%85SDK%E5%BC%80%E5%8F%91%E5%A5%97%E4%BB%B6.md) - -#### 1.3 代码目录结构与说明 -本sample工程名称为EdgeDetectionPicture，工程目录如下图所示： - -``` -. -├── model -│ ├── aipp.cfg // 模型转换aipp配置文件 -├── rcfDetection -│ ├── RcfDetection.cpp -│ └── RcfDetection.h -├── rcfPostProcess -│ ├── rcfPostProcess.cpp -│ └── rcfPostProcess.h -├── build.sh -├── main.cpp -├── README.md -├── CMakeLists.txt -└── License -``` - -## 2 环境依赖 -环境依赖软件和版本如下表： - - - -| 软件 | 版本 | 说明 | 获取方式 | -| ------------------- | ------------ | ----------------------------- | ------------------------------------------------------------ | -| mxVision | 5.0.0 | mxVision软件包 | [链接](https://www.hiascend.com/software/Mindx-sdk) | -| Ascend-CANN-toolkit | 7.0.0 | Ascend-cann-toolkit开发套件包 | [链接](https://www.hiascend.com/software/cann/commercial) | -| 操作系统 | Ubuntu 18.04 | 操作系统 | Ubuntu官网获取 | - -在编译运行项目前，需要设置环境变量： - -- 环境变量介绍 - - ``` - . {cann_install_path}/ascend-toolkit/set_env.sh - . {sdk_install_path}/mxVision/set_env.sh - - ``` - - - -## 3 模型转换 - -**步骤1** 模型获取 -下载RCF模型 。[下载地址](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/EdgeDetectionPicture/model.zip) - -**步骤2** 模型存放 -将获取到的RCF模型rcf.prototxt文件和rcf_bsds.caffemodel文件放在edge_detection_picture/model下 - -**步骤3** 执行模型转换命令 - -``` -atc --model=rcf.prototxt --weight=./rcf_bsds.caffemodel --framework=0 --output=rcf --soc_version=Ascend310 --insert_op_conf=./aipp.cfg --input_format=NCHW --output_type=FP32 -``` - -## 4 编译与运行 - -**步骤1** 执行如下编译命令： -bash build.sh - -**步骤2** 进行图像边缘检测 -请自行准备jpg格式的测试图像保存在文件夹中(例如 data/**.jpg)进行边缘检测 -``` -./edge_detection_picture ./data -``` -生成边缘检测图像 result/**.jpg - -## 5 精度测试 -下载开源数据集 BSDS500 [下载地址](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/EdgeDetectionPicture/data.zip), 使用 BSR/BSDS500/data/images/test数据进行测试 - - -(1) 下载开源代码 - -``` shell -git clone https://github.com/Walstruzz/edge_eval_python.git -cd edge_eval_python -``` -(2) 编译cxx - -``` shell -cd cxx/src -source build.sh -``` -(3) 将test中的图像经过边缘检测后的结果保存在result文件夹中 - -``` shell -./edge_detection_picture path/to/BSR/BSDS500/data/images/test/ - -``` - - -(4) 修改检测代码 - -vim mian.py -注释第17行代码 nms_process(model_name_list, result_dir, save_dir, key, file_format), -修改18行为 eval_edge(alg, model_name_list, result_dir, gt_dir, workers) - -vim /impl/edges_eval_dir.py -修改155行为 im = os.path.join(res_dir, "{}.jpg".format(i)) - -vim eval_edge.py -修改14行为 res_dir = result_dir - -(5) 测试精度 - -``` shell -python main.py --result_dir path/to/result --gt_dir paht/to/BSR/BSDS500/data/groundTruth/test - -``` -注: - result_dir: results directory - - gt_dir : ground truth directory - - -## 6常见问题 -### 6.1 精度测试脚本运行时, ModuleNotFoundError报错问题： -问题描述: -运行精度测试脚本过程中, 出现如下类似报错: -``` -********* -File "***/nms_process.py", line 6 in - from impl.toolbox import conv_tri, grad2 -ModuleNotFoundError: No module named 'impl.toolbox' -``` - -解决措施: -``` -方法一: -将环境变量PYTHONPATH里面的/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe去掉, 该路径根据cann实际安装位置不同可能会不一样,须自行确认 - -方法二: -执行命令: unset PYTHONPATH -``` \ No newline at end of file -- Gitee From 60e52abad429d3ce0ef7433007b1855297e474cd Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:16:42 +0000 Subject: [PATCH 12/27] =?UTF-8?q?=E6=9B=B4=E6=96=B0README?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/EdgeDetectionPicture/README.md | 156 +++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100644 contrib/EdgeDetectionPicture/README.md diff --git a/contrib/EdgeDetectionPicture/README.md b/contrib/EdgeDetectionPicture/README.md new file mode 100644 index 000000000..53642883f --- /dev/null +++ b/contrib/EdgeDetectionPicture/README.md @@ -0,0 +1,156 @@ + +# RCF模型边缘检测 + +## 1 介绍 + +#### 1.1 简介 +本开发样例是基于mxBase开发的端到端推理的C++应用程序，可在昇腾芯片上进行 图像边缘提取，并把可视化结果保存到本地。 +其中包含Rcf模型的后处理模块开发。 主要处理流程为： Init > ReadImage >Resize > Inference >PostProcess >DeInit + +#### 1.2 支持的产品 +昇腾310(推理) + +#### 1.3 支持的版本 +本样例配套的CANN版本为7.0.0，MindX SDK版本为5.0.0 +MindX SDK安装前准备可参考《用户指南》，[安装教程](https://gitee.com/ascend/mindxsdk-referenceapps/blob/master/docs/quickStart/1-1%E5%AE%89%E8%A3%85SDK%E5%BC%80%E5%8F%91%E5%A5%97%E4%BB%B6.md) +| 软件 | 版本 | 说明 | +| ------------------- | ------------ | ---------------------------- | +| mxVision | 5.0.0 | mxVision软件包 | +| Ascend-CANN-toolkit | 7.0.0 | Ascend-cann-toolkit开发套件包 | + + +#### 1.4 代码目录结构与说明 +本sample工程名称为EdgeDetectionPicture，工程目录如下图所示： + +``` +. +├── model +│ ├── aipp.cfg // 模型转换aipp配置文件 +├── rcfDetection +│ ├── RcfDetection.cpp +│ └── RcfDetection.h +├── rcfPostProcess +│ ├── rcfPostProcess.cpp +│ └── rcfPostProcess.h +├── build.sh +├── main.cpp +├── README.md +├── CMakeLists.txt +└── License +``` + +## 2 设置环境变量 + +在编译运行项目前，需要设置环境变量： + +- 环境变量介绍 + + ``` + . {cann_install_path}/ascend-toolkit/set_env.sh + . {sdk_install_path}/mxVision/set_env.sh + + ``` + + + +## 3 准备模型 + +**步骤1** 模型获取 +下载RCF模型 。[下载地址](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/EdgeDetectionPicture/model.zip) + +**步骤2** 模型存放 +将获取到的RCF模型rcf.prototxt文件和rcf_bsds.caffemodel文件放在edge_detection_picture/model下 + +**步骤3** 执行模型转换命令 + +``` +atc --model=rcf.prototxt --weight=./rcf_bsds.caffemodel --framework=0 --output=rcf --soc_version=Ascend310 --insert_op_conf=./aipp.cfg --input_format=NCHW --output_type=FP32 +``` + +## 4 编译与运行 + +**步骤1** 执行如下编译命令： +bash build.sh + +**步骤2** 进行图像边缘检测 +请自行准备jpg格式的测试图像保存在文件夹中(例如 data/**.jpg)进行边缘检测 +``` +./edge_detection_picture ./data +``` +生成边缘检测图像 result/**.jpg + +## 5 精度测试 +下载开源数据集 BSDS500 [下载地址](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/EdgeDetectionPicture/data.zip), 使用 BSR/BSDS500/data/images/test数据进行测试 + + +(1) 下载开源代码 + +``` shell +git clone https://github.com/Walstruzz/edge_eval_python.git +cd edge_eval_python +``` +(2) 编译cxx + +``` shell +cd cxx/src +source build.sh +``` +(3) 将test中的图像经过边缘检测后的结果保存在result文件夹中 + +``` shell +./edge_detection_picture path/to/BSR/BSDS500/data/images/test/ + +``` + + +(4) 修改检测代码 + +vim mian.py +注释第17行代码 nms_process(model_name_list, result_dir, save_dir, key, file_format), +修改18行为 eval_edge(alg, model_name_list, result_dir, gt_dir, workers) + +vim /impl/edges_eval_dir.py +修改155行为 im = os.path.join(res_dir, "{}.jpg".format(i)) + +vim eval_edge.py +修改14行为 res_dir = result_dir + +(5) 测试精度 + +``` shell +python main.py --result_dir path/to/result --gt_dir path/to/BSR/BSDS500/data/groundTruth/test + +``` +注: + result_dir: results directory + + gt_dir : ground truth directory + + +## 6常见问题 +### 6.1 精度测试脚本运行时, ModuleNotFoundError报错问题： +问题描述: +运行精度测试脚本过程中, 出现如下类似报错: +``` +********* +File "***/nms_process.py", line 6 in + from impl.toolbox import conv_tri, grad2 +ModuleNotFoundError: No module named 'impl.toolbox' +``` + +解决措施: +``` +方法一: +将环境变量PYTHONPATH里面的/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe去掉, 该路径根据cann实际安装位置不同可能会不一样,须自行确认 + +方法二: +执行命令: unset PYTHONPATH +``` +### 6.2 检测代码无法修改问题： +问题描述: +修改检测代码中, 出现无法修改问题 + +解决措施: +``` +使用 sudo vim filename进行修改 +``` \ No newline at end of file -- Gitee From 242855048bcea8e0784a24395ed259f284cbd873 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:17:02 +0000 Subject: [PATCH 13/27] =?UTF-8?q?=E6=96=B0=E5=BB=BA=20rcfDetection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/rcfDetection/.keep | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 contrib/EdgeDetectionPicture/rcfDetection/.keep diff --git a/contrib/EdgeDetectionPicture/rcfDetection/.keep b/contrib/EdgeDetectionPicture/rcfDetection/.keep new file mode 100644 index 000000000..e69de29bb -- Gitee From c38184b05b4a598af37167d258224b5a76718c8e Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:17:31 +0000 Subject: [PATCH 14/27] =?UTF-8?q?=E6=9B=B4=E6=96=B0RcfDetection=E4=BD=BF?= =?UTF-8?q?=E5=85=B6=E9=80=82=E9=85=8D=E7=89=88=E6=9C=AC?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- .../rcfDetection/RcfDetection.cpp | 286 ++++++++++++++++++ .../rcfDetection/RcfDetection.h | 61 ++++ 2 files changed, 347 insertions(+) create mode 100644 contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp create mode 100644 contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h diff --git a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp new file mode 100644 index 000000000..ab1796d72 --- /dev/null +++ b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp @@ -0,0 +1,286 @@ +/** + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "RcfDetection.h" +#include "MxBase/DeviceManager/DeviceManager.h" +#include "MxBase/Log/Log.h" +#include "../rcfPostProcess/RcfPostProcess.h" +#include +#include +#include "opencv2/opencv.hpp" + +namespace { + const uint32_t YUV_BYTE_NU = 3; + const uint32_t YUV_BYTE_DE = 2; + const uint32_t VPC_H_ALIGN = 2; + const uint32_t channel = 3; + const double ALPHA = 255.0 / 255; +} +void RcfDetection::SetRcfPostProcessConfig(const InitParam &initParam, + std::map &config) +{ + MxBase::ConfigData configData; + const std::string checkTensor = initParam.checkTensor ? "true" : "false"; + configData.SetJsonValue("OUTSIZE_NUM", std::to_string(initParam.outSizeNum)); + configData.SetJsonValue("OUTSIZE", initParam.outSize); + configData.SetJsonValue("RCF_TYPE", std::to_string(initParam.rcfType)); + configData.SetJsonValue("MODEL_TYPE", std::to_string(initParam.modelType)); + configData.SetJsonValue("INPUT_TYPE", std::to_string(initParam.inputType)); + configData.SetJsonValue("CHECK_MODEL", checkTensor); + #ifdef MX_VERSION_5 + auto jsonStr = configData.GetCfgJson().serialize(); + config["postProcessConfigContent"] = jsonStr; + #else + auto jsonStr = configData.GetCfgJson(); + config["postProcessConfigContent"] = jsonStr; + #endif +} + +APP_ERROR RcfDetection::Init(const InitParam &initParam) +{ + deviceId_ = initParam.deviceId; + APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices(); + if (ret != APP_ERR_OK) { + LogError << "Init devices failed, ret=" << ret << "."; + return ret; + } + ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId); + if (ret != APP_ERR_OK) { + LogError << "Set context failed, ret=" << ret << "."; + return ret; + } + dvppWrapper_ = std::make_shared(); + ret = dvppWrapper_->Init(); + if (ret != APP_ERR_OK) { + LogError << "DvppWrapper init failed, ret=" << ret << "."; + return ret; + } + model_ = std::make_shared(); + ret = model_->Init(initParam.modelPath, modelDesc_); + if (ret != APP_ERR_OK) { + LogError << "ModelInferenceProcessor init failed, ret=" << ret << "."; + return ret; + } + + std::map config; + SetRcfPostProcessConfig(initParam, config); + post_ = std::make_shared(); + ret = post_->Init(config); + if (ret != APP_ERR_OK) { + LogError << "RcfPostProcess init failed, ret=" << ret << "."; + return ret; + } + if (ret != APP_ERR_OK) { + LogError << "Failed to load labels, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::DeInit() +{ + dvppWrapper_->DeInit(); + model_->DeInit(); + post_->DeInit(); + MxBase::DeviceManager::GetInstance()->DestroyDevices(); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::ReadImage(const std::string &imgPath, + MxBase::TensorBase &tensor) +{ + MxBase::DvppDataInfo output = {}; + APP_ERROR ret = dvppWrapper_->DvppJpegDecode(imgPath, output); + if (ret != APP_ERR_OK) { + LogError << "DvppWrapper DvppJpegDecode failed, ret=" << ret << "."; + return ret; + } + MxBase::MemoryData memoryData((void*)output.data, output.dataSize, + MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + if (output.heightStride % VPC_H_ALIGN != 0) { + LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; + MxBase::MemoryHelper::MxbsFree(memoryData); + return APP_ERR_COMM_INVALID_PARAM; + } + dvppHeightStride = output.heightStride; + dvppWidthStride = output.widthStride; + std::vector shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; + tensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); + return APP_ERR_OK; +} +APP_ERROR RcfDetection::Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, + uint32_t resizeHeight, uint32_t resizeWidth) +{ + auto shape = inputTensor.GetShape(); + MxBase::DvppDataInfo input = {}; + input.height = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; + input.width = shape[1]; + input.heightStride = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; + input.widthStride = shape[1]; + input.dataSize = inputTensor.GetByteSize(); + input.data = (uint8_t*)inputTensor.GetBuffer(); + MxBase::ResizeConfig resize = {}; + resize.height = resizeHeight; + resize.width = resizeWidth; + MxBase::DvppDataInfo output = {}; + APP_ERROR ret = dvppWrapper_->VpcResize(input, output, resize); + if (ret != APP_ERR_OK) { + LogError << "VpcResize failed, ret=" << ret << "."; + return ret; + } + MxBase::MemoryData memoryData((void*)output.data, output.dataSize, + MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + if (output.heightStride % VPC_H_ALIGN != 0) { + LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; + MxBase::MemoryHelper::MxbsFree(memoryData); + return APP_ERR_COMM_INVALID_PARAM; + } + shape = {1, channel, output.heightStride, output.widthStride}; + outputTensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::Inference(const std::vector &inputs, + std::vector &outputs) +{ + std::vector output={}; + auto dtypes = model_->GetOutputDataType(); + for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) { + std::vector shape = {}; + for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) { + shape.push_back((uint32_t)modelDesc_.outputTensors[i].tensorDims[j]); + } + MxBase::TensorBase tensor(shape, dtypes[i], MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor); + if (ret != APP_ERR_OK) { + LogError << "TensorBaseMalloc failed, ret=" << ret << "."; + return ret; + } + outputs.push_back(tensor); + } + MxBase::DynamicInfo dynamicInfo = {}; + dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH; + auto startTime = std::chrono::high_resolution_clock::now(); + APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo); + auto endTime = std::chrono::high_resolution_clock::now(); + double costMs = std::chrono::duration(endTime - startTime).count(); + LogInfo<< "costMs:"<< costMs; + if (ret != APP_ERR_OK) { + LogError << "ModelInference failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::PostProcess(const MxBase::TensorBase &tensor, + const std::vector &outputs, + std::vector &postProcessOutput) +{ + auto shape = tensor.GetShape(); + MxBase::ResizedImageInfo imgInfo; + imgInfo.widthOriginal = shape[1]; + imgInfo.heightOriginal = shape[0] * YUV_BYTE_DE; + uint32_t widthResize = 512; + uint32_t heightResize = 512; + imgInfo.widthResize = widthResize; + imgInfo.heightResize = heightResize; + imgInfo.resizeType = MxBase::RESIZER_STRETCHING; + std::vector imageInfoVec = {}; + imageInfoVec.push_back(imgInfo); + APP_ERROR ret = post_->Process(outputs, postProcessOutput); + if (ret != APP_ERR_OK) { + LogError << "Process failed, ret=" << ret << "."; + return ret; + } + ret = post_->DeInit(); + if (ret != APP_ERR_OK) { + LogError << "RcfDetection DeInit failed"; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath) +{ + auto shape = inferTensor.GetShape(); + int dim2 = 2; + int dim3 = 3; + uint32_t height = shape[dim2]; + uint32_t width = shape[dim3]; + cv::Mat imgBgr = cv::imread(imgPath); + std::string fileName = imgPath.substr(imgPath.find_last_of("/") + 1); + uint32_t imageWidth = imgBgr.cols; + uint32_t imageHeight = imgBgr.rows; + cv::Mat modelOutput = cv::Mat(height, width, CV_32FC1, inferTensor.GetBuffer()); + cv::Mat grayMat; + cv::Mat resizedMat; + int crop = 5; + cv::Rect myROI(0, 0, imageWidth - crop, imageHeight); + resize(modelOutput, resizedMat, cv::Size(dvppWidthStride, dvppHeightStride), 0, 0, cv::INTER_LINEAR); + resizedMat.convertTo(grayMat, CV_8UC1, ALPHA); + cv::Mat croppedImage = grayMat(myROI); + resize(croppedImage, croppedImage, cv::Size(imageWidth, imageHeight), 0, 0, cv::INTER_LINEAR); + + std::string resultPathName = "result"; + if (access(resultPathName.c_str(), 0) != 0) { + int ret = mkdir(resultPathName.c_str(), S_IRUSR | S_IWUSR | S_IXUSR); + if (ret != 0) { + LogError << "Failed to create result directory: " << resultPathName << ", ret = " << ret; + return APP_ERR_COMM_FAILURE; + } + } + + cv::imwrite("./result/"+fileName, croppedImage); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::Process(const std::string &imgPath) +{ + MxBase::TensorBase inTensor; + APP_ERROR ret = ReadImage(imgPath, inTensor); + if (ret != APP_ERR_OK) { + LogError << "ReadImage failed, ret=" << ret << "."; + return ret; + } + MxBase::TensorBase outTensor; + uint32_t resizeHeight = 512; + uint32_t resizeWidth = 512; + ret = Resize(inTensor, outTensor, resizeHeight, resizeWidth); + if (ret != APP_ERR_OK) { + LogError << "Resize failed, ret=" << ret << "."; + return ret; + } + std::vector inputs = {}; + std::vector outputs = {}; + auto shape = outTensor.GetShape(); + inputs.push_back(outTensor); + ret = Inference(inputs, outputs); + if (ret != APP_ERR_OK) { + LogError << "Inference failed, ret=" << ret << "."; + return ret; + } + std::vector postProcessOutput={}; + ret = PostProcess(inTensor, outputs, postProcessOutput); + if (ret != APP_ERR_OK) { + LogError << "PostProcess failed, ret=" << ret << "."; + return ret; + } + ret = WriteResult(postProcessOutput[0], imgPath); + if (ret != APP_ERR_OK) { + LogError << "Save result failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} diff --git a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h new file mode 100644 index 000000000..7c0ae6da4 --- /dev/null +++ b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h @@ -0,0 +1,61 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef MXBASE_RCFDETECTION_H +#define MXBASE_RCFDETECTION_H + +#include +#include +#include "MxBase/DvppWrapper/DvppWrapper.h" +#include "MxBase/ModelInfer/ModelInferenceProcessor.h" +#include "MxBase/Tensor/TensorContext/TensorContext.h" + +struct InitParam { + uint32_t deviceId = 0; + bool checkTensor = true; + std::string modelPath = ""; + uint32_t outSizeNum = 0; + std::string outSize = ""; + uint32_t rcfType = 0; + uint32_t modelType = 0; + uint32_t inputType = 0; +}; + +class RcfDetection { +public: + APP_ERROR Init(const InitParam &initParam); + APP_ERROR DeInit(); + APP_ERROR Inference(const std::vector &inputs, std::vector &outputs); + APP_ERROR PostProcess(const MxBase::TensorBase &tensor, const std::vector &outputs, + std::vector &postProcessOutput); + APP_ERROR Process(const std::string &imgPath); +protected: + APP_ERROR ReadImage(const std::string &imgPath, MxBase::TensorBase &tensor); + APP_ERROR Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, + uint32_t resizeHeight, uint32_t resizeWidth); + APP_ERROR WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath); + void SetRcfPostProcessConfig(const InitParam &initParam, std::map &config); + +private: + std::shared_ptr dvppWrapper_; + std::shared_ptr model_; + std::shared_ptr post_; + MxBase::ModelDesc modelDesc_ = {}; + uint32_t deviceId_ = 0; + int dvppHeightStride = 0; + int dvppWidthStride = 0; +}; +#endif -- Gitee From 6bbe86b9a7c771bb196f2ff04f28c24f1a99123d Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:17:36 +0000 Subject: [PATCH 15/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/rcfDetection/.keep?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/rcfDetection/.keep | 0 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/rcfDetection/.keep diff --git a/contrib/EdgeDetectionPicture/rcfDetection/.keep b/contrib/EdgeDetectionPicture/rcfDetection/.keep deleted file mode 100644 index e69de29bb..000000000 -- Gitee From 6d94ab464e821fead4732ba4d0e45c6f0bd9bb50 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:17:45 +0000 Subject: [PATCH 16/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/RcfDetection.h?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/RcfDetection.h | 61 --------------------- 1 file changed, 61 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/RcfDetection.h diff --git a/contrib/EdgeDetectionPicture/RcfDetection.h b/contrib/EdgeDetectionPicture/RcfDetection.h deleted file mode 100644 index 7c0ae6da4..000000000 --- a/contrib/EdgeDetectionPicture/RcfDetection.h +++ /dev/null @@ -1,61 +0,0 @@ -/* - * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#ifndef MXBASE_RCFDETECTION_H -#define MXBASE_RCFDETECTION_H - -#include -#include -#include "MxBase/DvppWrapper/DvppWrapper.h" -#include "MxBase/ModelInfer/ModelInferenceProcessor.h" -#include "MxBase/Tensor/TensorContext/TensorContext.h" - -struct InitParam { - uint32_t deviceId = 0; - bool checkTensor = true; - std::string modelPath = ""; - uint32_t outSizeNum = 0; - std::string outSize = ""; - uint32_t rcfType = 0; - uint32_t modelType = 0; - uint32_t inputType = 0; -}; - -class RcfDetection { -public: - APP_ERROR Init(const InitParam &initParam); - APP_ERROR DeInit(); - APP_ERROR Inference(const std::vector &inputs, std::vector &outputs); - APP_ERROR PostProcess(const MxBase::TensorBase &tensor, const std::vector &outputs, - std::vector &postProcessOutput); - APP_ERROR Process(const std::string &imgPath); -protected: - APP_ERROR ReadImage(const std::string &imgPath, MxBase::TensorBase &tensor); - APP_ERROR Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, - uint32_t resizeHeight, uint32_t resizeWidth); - APP_ERROR WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath); - void SetRcfPostProcessConfig(const InitParam &initParam, std::map &config); - -private: - std::shared_ptr dvppWrapper_; - std::shared_ptr model_; - std::shared_ptr post_; - MxBase::ModelDesc modelDesc_ = {}; - uint32_t deviceId_ = 0; - int dvppHeightStride = 0; - int dvppWidthStride = 0; -}; -#endif -- Gitee From 092009ea6922a7d34c3d56db2ca0cdb7b8110205 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:17:51 +0000 Subject: [PATCH 17/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/RcfDetection.cpp?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/RcfDetection.cpp | 286 ------------------ 1 file changed, 286 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/RcfDetection.cpp diff --git a/contrib/EdgeDetectionPicture/RcfDetection.cpp b/contrib/EdgeDetectionPicture/RcfDetection.cpp deleted file mode 100644 index ab1796d72..000000000 --- a/contrib/EdgeDetectionPicture/RcfDetection.cpp +++ /dev/null @@ -1,286 +0,0 @@ -/** - * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#include "RcfDetection.h" -#include "MxBase/DeviceManager/DeviceManager.h" -#include "MxBase/Log/Log.h" -#include "../rcfPostProcess/RcfPostProcess.h" -#include -#include -#include "opencv2/opencv.hpp" - -namespace { - const uint32_t YUV_BYTE_NU = 3; - const uint32_t YUV_BYTE_DE = 2; - const uint32_t VPC_H_ALIGN = 2; - const uint32_t channel = 3; - const double ALPHA = 255.0 / 255; -} -void RcfDetection::SetRcfPostProcessConfig(const InitParam &initParam, - std::map &config) -{ - MxBase::ConfigData configData; - const std::string checkTensor = initParam.checkTensor ? "true" : "false"; - configData.SetJsonValue("OUTSIZE_NUM", std::to_string(initParam.outSizeNum)); - configData.SetJsonValue("OUTSIZE", initParam.outSize); - configData.SetJsonValue("RCF_TYPE", std::to_string(initParam.rcfType)); - configData.SetJsonValue("MODEL_TYPE", std::to_string(initParam.modelType)); - configData.SetJsonValue("INPUT_TYPE", std::to_string(initParam.inputType)); - configData.SetJsonValue("CHECK_MODEL", checkTensor); - #ifdef MX_VERSION_5 - auto jsonStr = configData.GetCfgJson().serialize(); - config["postProcessConfigContent"] = jsonStr; - #else - auto jsonStr = configData.GetCfgJson(); - config["postProcessConfigContent"] = jsonStr; - #endif -} - -APP_ERROR RcfDetection::Init(const InitParam &initParam) -{ - deviceId_ = initParam.deviceId; - APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices(); - if (ret != APP_ERR_OK) { - LogError << "Init devices failed, ret=" << ret << "."; - return ret; - } - ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId); - if (ret != APP_ERR_OK) { - LogError << "Set context failed, ret=" << ret << "."; - return ret; - } - dvppWrapper_ = std::make_shared(); - ret = dvppWrapper_->Init(); - if (ret != APP_ERR_OK) { - LogError << "DvppWrapper init failed, ret=" << ret << "."; - return ret; - } - model_ = std::make_shared(); - ret = model_->Init(initParam.modelPath, modelDesc_); - if (ret != APP_ERR_OK) { - LogError << "ModelInferenceProcessor init failed, ret=" << ret << "."; - return ret; - } - - std::map config; - SetRcfPostProcessConfig(initParam, config); - post_ = std::make_shared(); - ret = post_->Init(config); - if (ret != APP_ERR_OK) { - LogError << "RcfPostProcess init failed, ret=" << ret << "."; - return ret; - } - if (ret != APP_ERR_OK) { - LogError << "Failed to load labels, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::DeInit() -{ - dvppWrapper_->DeInit(); - model_->DeInit(); - post_->DeInit(); - MxBase::DeviceManager::GetInstance()->DestroyDevices(); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::ReadImage(const std::string &imgPath, - MxBase::TensorBase &tensor) -{ - MxBase::DvppDataInfo output = {}; - APP_ERROR ret = dvppWrapper_->DvppJpegDecode(imgPath, output); - if (ret != APP_ERR_OK) { - LogError << "DvppWrapper DvppJpegDecode failed, ret=" << ret << "."; - return ret; - } - MxBase::MemoryData memoryData((void*)output.data, output.dataSize, - MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - if (output.heightStride % VPC_H_ALIGN != 0) { - LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; - MxBase::MemoryHelper::MxbsFree(memoryData); - return APP_ERR_COMM_INVALID_PARAM; - } - dvppHeightStride = output.heightStride; - dvppWidthStride = output.widthStride; - std::vector shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; - tensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); - return APP_ERR_OK; -} -APP_ERROR RcfDetection::Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, - uint32_t resizeHeight, uint32_t resizeWidth) -{ - auto shape = inputTensor.GetShape(); - MxBase::DvppDataInfo input = {}; - input.height = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; - input.width = shape[1]; - input.heightStride = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; - input.widthStride = shape[1]; - input.dataSize = inputTensor.GetByteSize(); - input.data = (uint8_t*)inputTensor.GetBuffer(); - MxBase::ResizeConfig resize = {}; - resize.height = resizeHeight; - resize.width = resizeWidth; - MxBase::DvppDataInfo output = {}; - APP_ERROR ret = dvppWrapper_->VpcResize(input, output, resize); - if (ret != APP_ERR_OK) { - LogError << "VpcResize failed, ret=" << ret << "."; - return ret; - } - MxBase::MemoryData memoryData((void*)output.data, output.dataSize, - MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - if (output.heightStride % VPC_H_ALIGN != 0) { - LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; - MxBase::MemoryHelper::MxbsFree(memoryData); - return APP_ERR_COMM_INVALID_PARAM; - } - shape = {1, channel, output.heightStride, output.widthStride}; - outputTensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::Inference(const std::vector &inputs, - std::vector &outputs) -{ - std::vector output={}; - auto dtypes = model_->GetOutputDataType(); - for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) { - std::vector shape = {}; - for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) { - shape.push_back((uint32_t)modelDesc_.outputTensors[i].tensorDims[j]); - } - MxBase::TensorBase tensor(shape, dtypes[i], MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor); - if (ret != APP_ERR_OK) { - LogError << "TensorBaseMalloc failed, ret=" << ret << "."; - return ret; - } - outputs.push_back(tensor); - } - MxBase::DynamicInfo dynamicInfo = {}; - dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH; - auto startTime = std::chrono::high_resolution_clock::now(); - APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo); - auto endTime = std::chrono::high_resolution_clock::now(); - double costMs = std::chrono::duration(endTime - startTime).count(); - LogInfo<< "costMs:"<< costMs; - if (ret != APP_ERR_OK) { - LogError << "ModelInference failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::PostProcess(const MxBase::TensorBase &tensor, - const std::vector &outputs, - std::vector &postProcessOutput) -{ - auto shape = tensor.GetShape(); - MxBase::ResizedImageInfo imgInfo; - imgInfo.widthOriginal = shape[1]; - imgInfo.heightOriginal = shape[0] * YUV_BYTE_DE; - uint32_t widthResize = 512; - uint32_t heightResize = 512; - imgInfo.widthResize = widthResize; - imgInfo.heightResize = heightResize; - imgInfo.resizeType = MxBase::RESIZER_STRETCHING; - std::vector imageInfoVec = {}; - imageInfoVec.push_back(imgInfo); - APP_ERROR ret = post_->Process(outputs, postProcessOutput); - if (ret != APP_ERR_OK) { - LogError << "Process failed, ret=" << ret << "."; - return ret; - } - ret = post_->DeInit(); - if (ret != APP_ERR_OK) { - LogError << "RcfDetection DeInit failed"; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath) -{ - auto shape = inferTensor.GetShape(); - int dim2 = 2; - int dim3 = 3; - uint32_t height = shape[dim2]; - uint32_t width = shape[dim3]; - cv::Mat imgBgr = cv::imread(imgPath); - std::string fileName = imgPath.substr(imgPath.find_last_of("/") + 1); - uint32_t imageWidth = imgBgr.cols; - uint32_t imageHeight = imgBgr.rows; - cv::Mat modelOutput = cv::Mat(height, width, CV_32FC1, inferTensor.GetBuffer()); - cv::Mat grayMat; - cv::Mat resizedMat; - int crop = 5; - cv::Rect myROI(0, 0, imageWidth - crop, imageHeight); - resize(modelOutput, resizedMat, cv::Size(dvppWidthStride, dvppHeightStride), 0, 0, cv::INTER_LINEAR); - resizedMat.convertTo(grayMat, CV_8UC1, ALPHA); - cv::Mat croppedImage = grayMat(myROI); - resize(croppedImage, croppedImage, cv::Size(imageWidth, imageHeight), 0, 0, cv::INTER_LINEAR); - - std::string resultPathName = "result"; - if (access(resultPathName.c_str(), 0) != 0) { - int ret = mkdir(resultPathName.c_str(), S_IRUSR | S_IWUSR | S_IXUSR); - if (ret != 0) { - LogError << "Failed to create result directory: " << resultPathName << ", ret = " << ret; - return APP_ERR_COMM_FAILURE; - } - } - - cv::imwrite("./result/"+fileName, croppedImage); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::Process(const std::string &imgPath) -{ - MxBase::TensorBase inTensor; - APP_ERROR ret = ReadImage(imgPath, inTensor); - if (ret != APP_ERR_OK) { - LogError << "ReadImage failed, ret=" << ret << "."; - return ret; - } - MxBase::TensorBase outTensor; - uint32_t resizeHeight = 512; - uint32_t resizeWidth = 512; - ret = Resize(inTensor, outTensor, resizeHeight, resizeWidth); - if (ret != APP_ERR_OK) { - LogError << "Resize failed, ret=" << ret << "."; - return ret; - } - std::vector inputs = {}; - std::vector outputs = {}; - auto shape = outTensor.GetShape(); - inputs.push_back(outTensor); - ret = Inference(inputs, outputs); - if (ret != APP_ERR_OK) { - LogError << "Inference failed, ret=" << ret << "."; - return ret; - } - std::vector postProcessOutput={}; - ret = PostProcess(inTensor, outputs, postProcessOutput); - if (ret != APP_ERR_OK) { - LogError << "PostProcess failed, ret=" << ret << "."; - return ret; - } - ret = WriteResult(postProcessOutput[0], imgPath); - if (ret != APP_ERR_OK) { - LogError << "Save result failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} -- Gitee From 7a361e132d66248b60ff1a52f2a0473b351a51aa Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:18:44 +0000 Subject: [PATCH 18/27] update contrib/EdgeDetectionPicture/README.md. Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/EdgeDetectionPicture/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/contrib/EdgeDetectionPicture/README.md b/contrib/EdgeDetectionPicture/README.md index 53642883f..ec2f72b99 100644 --- a/contrib/EdgeDetectionPicture/README.md +++ b/contrib/EdgeDetectionPicture/README.md @@ -19,7 +19,7 @@ MindX SDK安装前准备可参考《用户指南》，[安装教程](https://git | Ascend-CANN-toolkit | 7.0.0 | Ascend-cann-toolkit开发套件包 | -#### 1.4 代码目录结构与说明 +#### 1.4 代码目录结构说明 本sample工程名称为EdgeDetectionPicture，工程目录如下图所示： ``` @@ -48,7 +48,6 @@ MindX SDK安装前准备可参考《用户指南》，[安装教程](https://git ``` . {cann_install_path}/ascend-toolkit/set_env.sh . {sdk_install_path}/mxVision/set_env.sh - ``` -- Gitee From 8b88b3b89adc14434e9f9dab9c94d462f111644c Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 01:53:46 +0000 Subject: [PATCH 19/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/TSM/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/TSM/README.md | 341 ------------------------------------------ 1 file changed, 341 deletions(-) delete mode 100644 contrib/TSM/README.md diff --git a/contrib/TSM/README.md b/contrib/TSM/README.md deleted file mode 100644 index 7356a5817..000000000 --- a/contrib/TSM/README.md +++ /dev/null @@ -1,341 +0,0 @@ -# TSM视频分类参考设计 - -## 1 介绍 -使用TSM模型，基于Kinetics-400数据集，在MindX SDK环境下实现视频分类功能。将测试视频传入脚本进行前处理，模型推理，后处理等功能，最终得到模型推理的精度和性能。 - -### 1.1 支持的产品 - -以昇腾Atlas310卡为主要的硬件平台 - -### 1.2 支持的版本 - -CANN：7.0.RC1 - -SDK：mxVision 5.0.RC3（可通过cat SDK目录下的 version.info 查看） - -### 1.3 软件方案介绍 - -项目主要由离线精度测试文件，在线功能测试文件，离线单视频推理性能测试文件，模型文件，测试数据集预处理文件组成。 - -### 1.4 代码目录结构与说明 - -```text -├── TSM - ├── README.md // 所有模型相关说明 - ├── model - ├── onnx2om.sh // 转om模型脚本 - ├── onnx2om1.sh // 在线模型转om模型脚本 - ├── label - ├── kinetics_val.csv // label文件 - ├── download_data - ├── k400_extractor.sh // 解压数据集脚本 - ├── offline.png // 离线推理技术实现流程 - ├── online.png // 在线推理技术实现流程 - ├── online_infer.py // 在线推理精度脚本 - ├── offline_infer.py // 离线推理精度脚本 - ├── speed.py // 离线单视频推理NPU性能脚本 - ├── speed_gpu.py // 离线单视频推理GPU性能脚本 -``` - -### 1.5技术实现流程 - -离线推理流程： - -![离线推理流程](./offline.png) - -在线推理流程： - -![离线推理流程](./online.png) - -### 1.6特性及适用场景 - -离线模型： - -本案例中的 TSM 模型适用于Kinetics数据集中的400类视频分类，并可以返回测试集视频的精度值及单视频识别的种类、性能。 - -在以下两种情况视频分类情况不太好：1. 视频长度过短（小于3s）。 2. 视频帧率过低。 - -在线模型： - -本案例中的在线模型适用于26中手势识别，并可以返回识别手势的名称。 - -## 2 环境依赖 - -推荐系统为ubuntu 18.04，环境依赖软件和版本如下表 - -| 软件名称 | 版本 | -|----------|--------| -| cmake | 3.5+ | -| mxVision | 5.0.RC3 | -| Python | 3.9 | -| torch | 1.10.0 | -| ffmpeg | 4.2.1 | - -- 环境变量搭建 - -在运行项目前，需要设置环境变量： - -MindSDK 环境变量： - -```Shell -. ${SDK-path}/set_env.sh -``` - -CANN 环境变量： - -```Shell -. ${ascend-toolkit-path}/set_env.sh -``` - -环境变量介绍 - -SDK-path: mxVision SDK 安装路径 - -ascend-toolkit-path: CANN 安装路径。 - -下载[ffmpeg](https://github.com/FFmpeg/FFmpeg/archive/n4.2.1.tar.gz)，解压进入并执行以下命令安装： - -```Shell -./configure --prefix=/usr/local/ffmpeg --enable-shared -make -j -make install -``` - -安装完毕后导入环境变量 - -```Shell -export PATH=/usr/local/ffmpeg/bin:$PATH -export LD_LIBRARY_PATH=/usr/local/ffmpeg/lib:$LD_LIBRARY_PATH -``` - -## 3 离线推理 - -**步骤1** Kinetics-400数据集下载 - -参考[Kinetics-400 数据准备](https://github.com/PaddlePaddle/PaddleVideo/blob/develop/docs/zh-CN/dataset/k400.md#%E4%B8%8B%E8%BD%BDvideo%E6%95%B0%E6%8D%AE)中的脚本下载操作，在代码根目录的"download_data"目录下准备"download.sh"数据集下载脚本和"val_link.list"验证集链接列表文件。 - -```text -├── TSM - ├── download_data - ├── download.sh // 下载数据集脚本 - ├── k400_extractor.sh // 解压数据集脚本 - ├── val_link.list -``` - -进入代码根目录的"download_data"目录下，执行以下命令下载数据集压缩包val_part1.tar、val_part2.tar、val_part3.tar： - -```Shell -bash download.sh val_link.list -``` - -然后执行以下命令解压数据集到代码根目录下： - -```Shell -bash k400_extractor.sh -``` - -数据集结构如下： - -```text -├── TSM - ├── data - ├── abseiling - ├── air_drumming - ├── ... - ├── zumba -``` - -**步骤2** 数据集预处理 - -1、视频抽帧 - -在代码根目录执行以下命令创建所需目录： - -```Shell -mkdir tools -mkdir ops -``` - -下载[“temporal-shift-module-master.zip”](https://github.com/mit-han-lab/temporal-shift-module/tree/master)代码包并上传服务器解压，将代码包中"tools"目录下的"vid2img_kinetics.py"、"gen_label_kinetics.py"、"kinetics_label_map.txt"三个文件拷贝至参考设计代码根目录的“tools”目录下。 - -```text -├── TSM - ├── tools - ├── gen_label_kinetics.py // label生成脚本 - ├── vid2img_kinetics.py // 视频抽帧脚本 - ├── kinetics_label_map.txt -``` - -将代码包中"ops"目录下的"basic_ops.py"、"dataset.py"、"dataset_config.py"、"models.py"、"temporal_shift.py"、"transforms.py"六个文件拷贝至参考设计代码根目录的“ops”目录下。 - -```text - ├── ops - ├── basic_ops.py - ├── dataset.py // 数据集构建脚本 - ├── dataset_config.py // 数据集配置脚本 - ├── models.py // 模型搭建脚本 - ├── temporal_shift.py - ├── transforms.py -``` - -修改“tools”目录下的 vid2img_kinetics.py 内容，将77、78行注释。 - -```text - -77行 #class_name = 'test' -78行 #class_process(dir_path, dst_dir_path, class_name) - -``` - -在参考设计代码根目录下，执行以下命令对数据集视频进行抽帧并生成图片： - -```shell -mkdir dataset -cd ./tools -python3 vid2img_kinetics.py [video_path] [image_path] -e.g. -python3 vid2img_kinetics.py ../data ../dataset/ -``` - -修改“tools”目录下gen_label_kinetics.py 内容。 - -```text - -# 11行 dataset_path = '../dataset' # 放视频抽帧后的图片路径 -# 12行 label_path = '../label' # 存放label路径 -# 25行 files_input = ['kinetics_val.csv'] -# 26行 files_output = ['val_videofolder.txt'] -# 37行 folders.append(items[1]) -# 57行 output.append('%s %d %d'%(os.path.join('../dataset/',os.path.join(categories_list[i], curFolder)), len(dir_files), curIDX)) - -``` - -在“tools”目录下，执行以下命令生成标签文件： - -```shell -python3 gen_label_kinetics.py -``` - -**步骤3** 模型转换 - -下载[离线模型](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/TSM/offline_models.zip) TSM.onnx, 将下载好的模型放在“${TSM代码根目录}/model”目录下。 - -将模型转换为om模型，在“model”目录下，执行以下命令生成om模型 - -```shell -bash onnx2om.sh -``` - -**步骤4** 精度测试 - -修改${TSM代码根目录}/ops/dataset_config.py 脚本中参数root_data、filename_imglist_train和filename_imglist_val，若仅进行离线精度测试则可忽略filename_imglist_train设置。 - -```shell -import os - -ROOT_DATASET = './labels/' # 标签文件所在路径 - -... - -def return_kinetics(modality): - filename_categories = 400 - if modality == 'RGB': - root_data = ROOT_DATASET # 训练集根目录 - filename_imglist_train = 'train_videofolder.txt' # 训练数据集标签 - filename_imglist_val = 'val_videofolder.txt' # 测试数据集标签 - prefix = 'img_{:05d}.jpg' - else: - raise NotImplementedError('no such modality:' + modality) - return filename_categories, filename_imglist_train, filename_imglist_val, root_data, prefix -``` - -在参考设计代码根目录下，运行精度测试脚本 - -```shell -python3 offline_infer.py kinetics -``` - -原模型精度值为71.1%，实测精度值为71.01%，符合精度偏差范围，精度达标。 - -**步骤5** 性能测试 - -将用来测试的单视频放在参考设计代码根目录下，如视频“test_speed.mp4”，运行性能测试脚本 - -修改speed_gpu.py与speed.py参数，'./test_speed.mp4'为测试视频，测试视频类别需在Kinetics-400数据集的400个种类内且视频长度至少为3s。 - -```python -def main(): - cmd = 'ffmpeg -i \"{}\" -threads 1 -vf scale=-1:331 -q:v 0 \"{}/img_%05d.jpg\"'.format('./test_speed.mp4', './image') - subprocess.call(cmd, shell=True, - stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) - files = os.listdir(r"./image/") -``` - -GPU性能(Tesla_V100S_PCIE_32GB) - -在参考设计代码根目录下，运行GPU性能测试脚本 - -```shell -python3 speed_gpu.py kinetics --test_segments=8 --test_crops=1 --batch_size=1 -``` - -注：speed_gpu.py脚本需在GPU环境上运行，NPU环境无法运行。 - -得到单视频纯推理性能为0.08sec/video - -SDK性能 - -在参考设计代码根目录下，运行SDK性能测试脚本 - -```shell -python3 speed.py -``` - -注：speed.py脚本需在NPU环境上运行。 - -得到单视频纯推理性能为0.189sec/video - -## 4 在线推理 - -**步骤1** 安装[视频流工具](https://gitee.com/ascend/docs-openmind/blob/master/guide/mindx/sdk/tutorials/reference_material/Live555%E7%A6%BB%E7%BA%BF%E8%A7%86%E9%A2%91%E8%BD%ACRTSP%E8%AF%B4%E6%98%8E%E6%96%87%E6%A1%A3.md) - -**步骤2** 生成视频流 - -根据提示当前只支持部分视频格式，并不支持.mp4后缀的文件，但可以通过ffmpeg转换生成[ffmpeg安装教程](https://gitee.com/ascend/docs-openmind/blob/master/guide/mindx/sdk/tutorials/reference_material/pc%E7%AB%AFffmpeg%E5%AE%89%E8%A3%85%E6%95%99%E7%A8%8B.md)，如下所示为MP4转换为h.264命令： - -使用ffmpeg工具将带有手势的“jester.mp4”的mp4格式视频转换生成为“jester.264”的264格式视频： - -```shell -ffmpeg -i jester.mp4 -vcodec h264 -bf 0 -g 25 -r 10 -s 1280*720 -an -f h264 jester.264 - -//-bf B帧数目控制，-g 关键帧间隔控制，-s 分辨率控制 -an关闭音频， -r 指定帧率 -``` - -使用live555生成视频流。 - -**步骤3** 模型转换 - -下载[在线模型](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/TSM/online_models.zip) jester.onnx - -将下载好的模型放在参考设计代码根目录的“model”目录下。 - -将模型转换为om模型，在“model”目录下，运行脚本生成om模型 - -```shell -bash onnx2om1.sh -``` - -**步骤4** 程序测试 - -```shell -python3 online_infer.py -``` - -修改参数，'ip:port/jester.264'为测试视频流，其中ip为起流的机器ip地址，port为起流的机器端口地址，jester.264为测试视频jester.mp4通过ffmpeg转换后的视频。 - -```python -def video2img(): - cmd = 'ffmpeg -i \"{}\" -threads 1 -vf scale=-1:331 -q:v 0 \"{}/img_%05d.jpg\"'.format('rtsp://ip:port/jester.264', './image') - subprocess.call(cmd, shell=True, - stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) -``` -- Gitee From ba11b674371f0a92621ccbca8ca3e9e9b45a51ce Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 01:54:00 +0000 Subject: [PATCH 20/27] =?UTF-8?q?=E6=9B=B4=E6=96=B0README?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/TSM/README.md | 141 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 contrib/TSM/README.md diff --git a/contrib/TSM/README.md b/contrib/TSM/README.md new file mode 100644 index 000000000..cc2c0f52b --- /dev/null +++ b/contrib/TSM/README.md @@ -0,0 +1,141 @@ +# TSM视频分类参考设计 + +## 1 介绍 + +### 1.1 简介 +使用TSM模型，在MindX SDK环境下实现视频分类功能。将测试视频传入脚本进行前处理，模型推理，后处理等功能，最终得到模型推理的结果。 + +### 1.2 支持的产品 + +以昇腾Atlas310卡为主要的硬件平台 + +### 1.3 支持的版本 +环境依赖软件和版本如下表： + +| 软件名称 | 版本 | +|----------|--------| +| cmake | 3.5+ | +| mxVision | 5.0.RC3 | +| Python | 3.9 | +| torch | 1.10.0 | +| ffmpeg | 4.2.1 | + +### 1.4 软件方案介绍 + +项目主要由离线精度测试文件，在线功能测试文件，离线单视频推理性能测试文件，模型文件，测试数据集预处理文件组成。 + +### 1.5 代码目录结构与说明 + +```text +├── TSM + ├── README.md // 所有模型相关说明 + ├── model + ├── onnx2om.sh // 转om模型脚本 + ├── onnx2om1.sh // 在线模型转om模型脚本 + ├── label + ├── kinetics_val.csv // label文件 + ├── download_data + ├── k400_extractor.sh // 解压数据集脚本 + ├── offline.png // 离线推理技术实现流程 + ├── online.png // 在线推理技术实现流程 + ├── online_infer.py // 在线推理精度脚本 + ├── offline_infer.py // 离线推理精度脚本 + ├── speed.py // 离线单视频推理NPU性能脚本 + ├── speed_gpu.py // 离线单视频推理GPU性能脚本 +``` + +### 1.6技术实现流程 + +在线推理流程： + +![离线推理流程](./online.png) + +### 1.7特性及适用场景 + +在线模型： + +本案例中的在线模型适用于26中手势识别，并可以返回识别手势的名称。 + +## 2 设置环境变量 + +- 环境变量搭建 + +在运行项目前，需要设置环境变量： + +MindSDK 环境变量： + +```Shell +. ${SDK-path}/set_env.sh +``` + +CANN 环境变量： + +```Shell +. ${ascend-toolkit-path}/set_env.sh +``` + +环境变量介绍 + +SDK-path: mxVision SDK 安装路径 + +ascend-toolkit-path: CANN 安装路径。 + +下载[ffmpeg](https://github.com/FFmpeg/FFmpeg/archive/n4.2.1.tar.gz)，解压进入并执行以下命令安装： + +```Shell +./configure --prefix=/usr/local/ffmpeg --enable-shared +make -j +make install +``` + +安装完毕后导入环境变量 + +```Shell +export PATH=/usr/local/ffmpeg/bin:$PATH +export LD_LIBRARY_PATH=/usr/local/ffmpeg/lib:$LD_LIBRARY_PATH +``` + +## 3 在线推理 + +**步骤1** 安装[视频流工具](https://gitee.com/ascend/docs-openmind/blob/master/guide/mindx/sdk/tutorials/reference_material/Live555%E7%A6%BB%E7%BA%BF%E8%A7%86%E9%A2%91%E8%BD%ACRTSP%E8%AF%B4%E6%98%8E%E6%96%87%E6%A1%A3.md) + +**步骤2** 生成视频流 + +根据提示当前只支持部分视频格式，并不支持.mp4后缀的文件，但可以通过ffmpeg转换生成[ffmpeg安装教程](https://gitee.com/ascend/docs-openmind/blob/master/guide/mindx/sdk/tutorials/reference_material/pc%E7%AB%AFffmpeg%E5%AE%89%E8%A3%85%E6%95%99%E7%A8%8B.md)，如下所示为MP4转换为h.264命令： + +使用ffmpeg工具将带有手势的“jester.mp4”的mp4格式视频转换生成为“jester.264”的264格式视频： + +```shell +ffmpeg -i jester.mp4 -vcodec h264 -bf 0 -g 25 -r 10 -s 1280*720 -an -f h264 jester.264 + +//-bf B帧数目控制，-g 关键帧间隔控制，-s 分辨率控制 -an关闭音频， -r 指定帧率 +``` + +使用live555生成视频流。 + +**步骤3** 模型转换 + +下载[在线模型](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/TSM/online_models.zip) jester.onnx + +将下载好的模型放在参考设计代码根目录的“model”目录下。 + +将模型转换为om模型，在“model”目录下，运行脚本生成om模型 + +```shell +bash onnx2om1.sh +``` + +**步骤4** 程序测试 + +```shell +python3 online_infer.py +``` + +修改参数，'ip:port/jester.264'为测试视频流，其中ip为起流的机器ip地址，port为起流的机器端口地址，jester.264为测试视频jester.mp4通过ffmpeg转换后的视频。 + +```python +def video2img(): + cmd = 'ffmpeg -i \"{}\" -threads 1 -vf scale=-1:331 -q:v 0 \"{}/img_%05d.jpg\"'.format('rtsp://ip:port/jester.264', './image') + subprocess.call(cmd, shell=True, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) +``` -- Gitee From 2deb845caece73511497b7de04e631f5a47624bd Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 02:42:01 +0000 Subject: [PATCH 21/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/CrowdCounting/CMakeLists.txt?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/CrowdCounting/CMakeLists.txt | 34 ---------------------------- 1 file changed, 34 deletions(-) delete mode 100644 contrib/CrowdCounting/CMakeLists.txt diff --git a/contrib/CrowdCounting/CMakeLists.txt b/contrib/CrowdCounting/CMakeLists.txt deleted file mode 100644 index 00606d306..000000000 --- a/contrib/CrowdCounting/CMakeLists.txt +++ /dev/null @@ -1,34 +0,0 @@ -cmake_minimum_required(VERSION 3.10) -project(crowd_counting) - -include_directories(./CrowdCountPostProcess) -include_directories(./CrowdCount) -file(GLOB_RECURSE CROWDCOUNT_POSTPROCESS ${PROJECT_SOURCE_DIR}/CrowdCountPostProcess/*cpp) -file(GLOB_RECURSE CROWDCOUNT ${PROJECT_SOURCE_DIR}/CrowdCount/*cpp) -set(TARGET crowd_counting) -add_compile_options(-std=c++11 -fPIC -fstack-protector-all -g -Wl,-z,relro,-z,now,-z -pie -Wall) - -add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) - -set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") - -include_directories( - ${MX_SDK_HOME}/include - ${MX_SDK_HOME}/opensource/include - ${MX_SDK_HOME}/opensource/include/opencv4 -) - -link_directories( - ${MX_SDK_HOME}/lib - ${MX_SDK_HOME}/opensource/lib - ${MX_SDK_HOME}/lib/modelpostprocessors - ${MX_SDK_HOME}/include/MxBase/postprocess/include -) - -add_executable(${TARGET} main.cpp ${CROWDCOUNT} ${CROWDCOUNT_POSTPROCESS}) -target_link_libraries(${TARGET} - glog - mxbase - opencv_world - ) -install(TARGETS ${TARGET} RUNTIME DESTINATION ${PROJECT_SOURCE_DIR}/) -- Gitee From 3382121b1307fd7d467e4aed5e2157ec7e6d6bbd Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 02:42:08 +0000 Subject: [PATCH 22/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/CrowdCounting/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/CrowdCounting/README.md | 107 -------------------------------- 1 file changed, 107 deletions(-) delete mode 100644 contrib/CrowdCounting/README.md diff --git a/contrib/CrowdCounting/README.md b/contrib/CrowdCounting/README.md deleted file mode 100644 index 00ba8047d..000000000 --- a/contrib/CrowdCounting/README.md +++ /dev/null @@ -1,107 +0,0 @@ -# C++ 基于MxBase 的人群计数图像检测样例及后处理模块开发 - -## 1 介绍 -本开发样例是基于mxBase开发的端到端推理的C++应用程序，可在昇腾芯片上进行人群计数目标检测，并把可视化结果保存到本地。开发端到端人群计数-人群密度估计的参考设计，实现图像当中人计数的功能，并达到精度和性能要求。 该Sample的主要处理流程为： Init > ReadImage >Resize > Inference >PostProcess >DeInit - -### 1.1 支持的产品 - -支持昇腾310芯片 - -### 1.2 支持的版本 - -在Atlas产品环境下，通过运行命令： - -``` -npu-smi info -``` - -可以查询支持SDK的版本号 - -### 1.3 软件方案介绍 - -人群计数项目实现：输入类型是图片数据（例如jpg格式的图片），通过调用MindX SDK mxBase提供的接口，使用DVPP进行图像解码，解码后获取图像数据，然后经过图像尺寸大小变换，满足模型的输入尺寸要求；将尺寸变换后的图像数据输入人群计数模型进行推理，模型输出经过后处理后，得到人群密度估计图和人计数估计值，输出人计数的估计值。 - -整个流程需要参考Ascend的参考样例：crowd_count_picture 样例，详见以下链接：https://gitee.com/ascend/samples/tree/master/python/contrib/crowd_count_picture crowd_count_picture 样例是基于ACL实现的，本任务需要参考crowd_count_picture 样例，基于MindX SDK mxBase的接口实现。MindX SDK mxBase是对ACL接口的封装，提供了极简易用的API， 使能AI应用的开发。 - -表1.1 系统方案中各模块功能： - -| 序号 | 子系统 | 功能描述 | -| ---- | ----------------- | ------------------------------------------------------------ | -| 1 | 设备初始化 | 调用mxBase::DeviceManager接口完成推理卡设备的初始化。 | -| 2 | 图像输入 | C++文件IO读取图像文件 | -| 3 | 图像解码/图像缩放 | 调用mxBase::DvppWrappe.DvppJpegDecode()函数完成图像解码，VpcResize()完成缩放。 | -| 4 | 模型推理 | 调用mxBase:: ModelInferenceProcessor 接口完成模型推理 | -| 5 | 后处理 | 获取模型推理输出张量BaseTensor，进行后处理。 | -| 6 | 保存结果 | 输出图像当中的人的数量，保存标记出人数的结果图像。 | -| 7 | 设备去初始化 | 调用mxBase::DeviceManager接口完成推理卡设备的去初始化。 | - -### 1.4 代码目录结构与说明 - -本sample工程名称为 **CrowdCounting**，工程目录如下图所示： - -![image-20210813152252629](image-20210813152252629.png) - -### 1.5 技术实现流程图 - -![image-20210813154111508](image-20210813154111508.png) - -## 2 环境依赖： - -请列出环境依赖软件和版本。 - -eg：推荐系统为ubuntu 18.04或centos 7.6，环境依赖软件和版本如下表： - -| 软件名称 | 版本 | -| -------- | ------------ | -| 系统软件 | ubuntu 18.04 | - -在编译运行项目前，需要设置环境变量： - - -``` -. {cann_install_path}/ascend-toolkit/set_env.sh -. {sdk_install_path}/mxVision/set_env.sh -``` - -## 3 模型转换 - -**步骤1** - -下载原始模型权重、原始模型网络、对应的cfg文件： - -[模型及配置文件下载链接](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/CrowdCounting/model.zip) - -**步骤2** - -将上述下载得文件统一放在CrowdCounting/model文件夹下，并使用ATC模型转换工具进行模型转换，参考如下指令: - -``` -atc --input_shape="blob1:1,3,800,1408" --weight="count_person.caffe.caffemodel" --input_format=NCHW --output="count_person.caffe" --soc_version=Ascend310 --insert_op_conf=insert_op.cfg --framework=0 --model="count_person.caffe.prototxt" -``` - -得到count_person.caffe.om文件 - -## 4 编译与运行 - -**步骤1** - -cd到CrowdCounting目录下，执行如下编译命令： bash build.sh - - -**步骤2** - -下载人群计数图像，需自行在网络找图，暂支持JPG格式，任意图像分辨率。再将该下载的图片作为推理图片放入CrowdCounting目录下，执行： - -``` -./crowd_counting ./xxx.jpg -``` - -## 5 软件依赖说明 - -| 依赖软件 | 版本 | 说明 | -| -------- | ----- | ------------------------------------------------------------ | -| mxVision | 5.0.0 | 封装昇腾计算语言(AscendCL)的高级编程API，简化插件和推理应用开发。 | -| CANN | 7.0.0 | 提供昇腾计算语言(AscendCL)的高级编程API。 | - - - -- Gitee From 7c5a29d4f5e8f7816bb2512e68b3d72ab775088d Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 02:42:27 +0000 Subject: [PATCH 23/27] =?UTF-8?q?=E9=80=82=E9=85=8D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/CrowdCounting/CMakeLists.txt | 41 ++++++++++++ contrib/CrowdCounting/README.md | 94 ++++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 contrib/CrowdCounting/CMakeLists.txt create mode 100644 contrib/CrowdCounting/README.md diff --git a/contrib/CrowdCounting/CMakeLists.txt b/contrib/CrowdCounting/CMakeLists.txt new file mode 100644 index 000000000..11c34fd96 --- /dev/null +++ b/contrib/CrowdCounting/CMakeLists.txt @@ -0,0 +1,41 @@ +cmake_minimum_required(VERSION 3.10) +project(crowd_counting) + +include_directories(./CrowdCountPostProcess) +include_directories(./CrowdCount) +file(GLOB_RECURSE CROWDCOUNT_POSTPROCESS ${PROJECT_SOURCE_DIR}/CrowdCountPostProcess/*cpp) +file(GLOB_RECURSE CROWDCOUNT ${PROJECT_SOURCE_DIR}/CrowdCount/*cpp) +set(TARGET crowd_counting) +add_compile_options(-std=c++11 -fPIC -fstack-protector-all -g -Wl,-z,relro,-z,now,-z -pie -Wall) + +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) + +set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") + +set(cpprest_DIR ${MX_SDK_HOME}/opensource/lib/libcpprest.so) +if(EXISTS ${cpprest_DIR}) + target_link_libraries(crowd_counting cpprest) + add_definitions(-DMX_VERSION_5) +endif() + + +include_directories( + ${MX_SDK_HOME}/include + ${MX_SDK_HOME}/opensource/include + ${MX_SDK_HOME}/opensource/include/opencv4 +) + +link_directories( + ${MX_SDK_HOME}/lib + ${MX_SDK_HOME}/opensource/lib + ${MX_SDK_HOME}/lib/modelpostprocessors + ${MX_SDK_HOME}/include/MxBase/postprocess/include +) + +add_executable(${TARGET} main.cpp ${CROWDCOUNT} ${CROWDCOUNT_POSTPROCESS}) +target_link_libraries(${TARGET} + glog + mxbase + opencv_world + ) +install(TARGETS ${TARGET} RUNTIME DESTINATION ${PROJECT_SOURCE_DIR}/) diff --git a/contrib/CrowdCounting/README.md b/contrib/CrowdCounting/README.md new file mode 100644 index 000000000..a7959de12 --- /dev/null +++ b/contrib/CrowdCounting/README.md @@ -0,0 +1,94 @@ +# C++ 基于MxBase 的人群计数图像检测样例及后处理模块开发 + +## 1 介绍 + +### 1.1 简介 +本开发样例是基于mxBase开发的端到端推理的C++应用程序，可在昇腾芯片上进行人群计数目标检测，并把可视化结果保存到本地。开发端到端人群计数-人群密度估计的参考设计，实现图像当中人计数的功能，并达到精度和性能要求。 该Sample的主要处理流程为： Init > ReadImage >Resize > Inference >PostProcess >DeInit + +### 1.2 支持的产品 + +支持昇腾310芯片 + +### 1.3 支持的版本 + +环境依赖软件和版本如下表： + +表1.1 系统方案中各模块功能： + +| 软件名称 | 版本 | 说明 | +| -------- | ----- | ------------------------------------------------------------ | +| mxVision | 5.0.0 | 封装昇腾计算语言(AscendCL)的高级编程API，简化插件和推理应用开发。 | +| CANN | 7.0.0 | 提供昇腾计算语言(AscendCL)的高级编程API。 | + +### 1.4 软件方案介绍 + +人群计数项目实现：输入类型是图片数据（例如jpg格式的图片），通过调用MindX SDK mxBase提供的接口，使用DVPP进行图像解码，解码后获取图像数据，然后经过图像尺寸大小变换，满足模型的输入尺寸要求；将尺寸变换后的图像数据输入人群计数模型进行推理，模型输出经过后处理后，得到人群密度估计图和人计数估计值，输出人计数的估计值。 + +整个流程需要参考Ascend的参考样例：crowd_count_picture 样例，详见以下链接：https://gitee.com/ascend/samples/tree/master/python/contrib/crowd_count_picture crowd_count_picture 样例是基于ACL实现的，本任务需要参考crowd_count_picture 样例，基于MindX SDK mxBase的接口实现。MindX SDK mxBase是对ACL接口的封装，提供了极简易用的API， 使能AI应用的开发。 + +表1.2 系统方案中各模块功能： + +| 序号 | 子系统 | 功能描述 | +| ---- | ----------------- | ------------------------------------------------------------ | +| 1 | 设备初始化 | 调用mxBase::DeviceManager接口完成推理卡设备的初始化。 | +| 2 | 图像输入 | C++文件IO读取图像文件 | +| 3 | 图像解码/图像缩放 | 调用mxBase::DvppWrappe.DvppJpegDecode()函数完成图像解码，VpcResize()完成缩放。 | +| 4 | 模型推理 | 调用mxBase:: ModelInferenceProcessor 接口完成模型推理 | +| 5 | 后处理 | 获取模型推理输出张量BaseTensor，进行后处理。 | +| 6 | 保存结果 | 输出图像当中的人的数量，保存标记出人数的结果图像。 | +| 7 | 设备去初始化 | 调用mxBase::DeviceManager接口完成推理卡设备的去初始化。 | + +### 1.5 代码目录结构与说明 + +本sample工程名称为 **CrowdCounting**，工程目录如下图所示： + +![image-20210813152252629](image-20210813152252629.png) + +### 1.6 技术实现流程图 + +![image-20210813154111508](image-20210813154111508.png) + +## 2 设置环境变量： + +在编译运行项目前，需要设置环境变量： + + +``` +. {cann_install_path}/ascend-toolkit/set_env.sh +. {sdk_install_path}/mxVision/set_env.sh +``` + +## 3 准备模型 + +**步骤1** + +下载原始模型权重、原始模型网络、对应的cfg文件： + +[模型及配置文件下载链接](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/CrowdCounting/model.zip) + +**步骤2** + +将上述下载得文件统一放在CrowdCounting/model文件夹下，并使用ATC模型转换工具进行模型转换，参考如下指令: + +``` +atc --input_shape="blob1:1,3,800,1408" --weight="count_person.caffe.caffemodel" --input_format=NCHW --output="count_person.caffe" --soc_version=Ascend310 --insert_op_conf=insert_op.cfg --framework=0 --model="count_person.caffe.prototxt" +``` + +得到count_person.caffe.om文件 + +## 4 编译与运行 + +**步骤1** + +cd到CrowdCounting目录下，执行如下编译命令： bash build.sh + + +**步骤2** + +下载人群计数图像，需自行在网络找图，暂支持JPG格式，任意图像分辨率。再将该下载的图片作为推理图片放入CrowdCounting目录下，执行： + +``` +./crowd_counting ./xxx.jpg +``` + + -- Gitee From 41dd195f13f1b9ffd7c3ad3c765c4834034993a6 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 02:42:34 +0000 Subject: [PATCH 24/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/CrowdCounting/CrowdCount/CrowdCount.cpp?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../CrowdCounting/CrowdCount/CrowdCount.cpp | 265 ------------------ 1 file changed, 265 deletions(-) delete mode 100644 contrib/CrowdCounting/CrowdCount/CrowdCount.cpp diff --git a/contrib/CrowdCounting/CrowdCount/CrowdCount.cpp b/contrib/CrowdCounting/CrowdCount/CrowdCount.cpp deleted file mode 100644 index ea0bb6c1f..000000000 --- a/contrib/CrowdCounting/CrowdCount/CrowdCount.cpp +++ /dev/null @@ -1,265 +0,0 @@ -/* - * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#include "opencv2/opencv.hpp" -#include "CrowdCount.h" -#include "MxBase/DeviceManager/DeviceManager.h" -#include "MxBase/Log/Log.h" -#include -#include -#include - -namespace{ - const uint32_t YUV_BYTE_NU = 3; - const uint32_t YUV_BYTE_DE = 2; - const uint32_t VPC_H_ALIGN = 2; - const uint32_t RESIZE_WIDTH = 1408; - const uint32_t RESIZE_HEIGHT = 800; -} -// 设置配置参数 -void CrowdCount::SetCrowdCountPostProcessConfig(const InitParam &initParam, - std::map &config) { - MxBase::ConfigData configData; - const std::string checkTensor = initParam.checkTensor ? "true" : "false"; - configData.SetJsonValue("CHECK_MODEL", checkTensor); - configData.SetJsonValue("CLASS_NUM", std::to_string(initParam.classNum)); - auto jsonStr = configData.GetCfgJson().serialize(); - config["postProcessConfigContent"] = jsonStr; -} - -APP_ERROR CrowdCount::Init(const InitParam &initParam) { - deviceId_ = initParam.deviceId; - APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices(); - if (ret != APP_ERR_OK) { - LogError << "Init devices failed, ret=" << ret << "."; - return ret; - } - ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId); - if (ret != APP_ERR_OK) { - LogError << "Set context failed, ret=" << ret << "."; - return ret; - } - dvppWrapper_ = std::make_shared(); - ret = dvppWrapper_->Init(); - if (ret != APP_ERR_OK) { - LogError << "DvppWrapper init failed, ret=" << ret << "."; - return ret; - } - model_ = std::make_shared(); - ret = model_->Init(initParam.modelPath, modelDesc_); - if (ret != APP_ERR_OK) { - LogError << "ModelInferenceProcessor init failed, ret=" << ret << "."; - return ret; - } - std::map config; - SetCrowdCountPostProcessConfig(initParam, config); - // 初始化CrowdCount后处理对象 - post_ = std::make_shared(); - ret = post_->Init(config); - if (ret != APP_ERR_OK) { - LogError << "CrowdCountstProcess init failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR CrowdCount::DeInit() { - dvppWrapper_->DeInit(); - model_->DeInit(); - post_->DeInit(); - MxBase::DeviceManager::GetInstance()->DestroyDevices(); - return APP_ERR_OK; -} -// 调用mxBase::DvppWrappe.DvppJpegDecode()函数完成图像解码，VpcResize()完成缩放 -APP_ERROR CrowdCount::ReadImage(const std::string &imgPath, MxBase::TensorBase &tensor) { - MxBase::DvppDataInfo output = {}; - APP_ERROR ret = dvppWrapper_->DvppJpegDecode(imgPath, output); - if (ret != APP_ERR_OK) { - LogError << "DvppWrapper DvppJpegDecode failed, ret=" << ret << "."; - return ret; - } - // 将数据转为到DEVICE侧，以便后续处理 - MxBase::MemoryData memoryData((void*)output.data, output.dataSize, - MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - // 对解码后图像对齐尺寸进行判定 - if (output.heightStride % VPC_H_ALIGN != 0) { - LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; - MxBase::MemoryHelper::MxbsFree(memoryData); - return APP_ERR_COMM_INVALID_PARAM; - } - std::vector shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; - tensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); - return APP_ERR_OK; -} - -APP_ERROR CrowdCount::Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor){ - auto shape = inputTensor.GetShape(); - // 还原为原始尺寸 - MxBase::DvppDataInfo input = {}; - input.height = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; - input.width = shape[1]; - input.heightStride = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; - input.widthStride = shape[1]; - input.dataSize = inputTensor.GetByteSize(); - input.data = (uint8_t*)inputTensor.GetBuffer(); - MxBase::ResizeConfig resize = {}; - resize.height = RESIZE_HEIGHT; - resize.width = RESIZE_WIDTH; - MxBase::DvppDataInfo output = {}; - // 图像缩放 - APP_ERROR ret = dvppWrapper_->VpcResize(input, output, resize); - if (ret != APP_ERR_OK) { - LogError << "VpcResize failed, ret=" << ret << "."; - return ret; - } - // 对缩放后图像对齐尺寸进行判定 - MxBase::MemoryData memoryData((void*)output.data, output.dataSize, - MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - if (output.heightStride % VPC_H_ALIGN != 0) { - LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; - MxBase::MemoryHelper::MxbsFree(memoryData); - return APP_ERR_COMM_INVALID_PARAM; - } - shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; - outputTensor = MxBase::TensorBase(memoryData, false, shape,MxBase::TENSOR_DTYPE_UINT8); - return APP_ERR_OK; - -} -// 模型推理 -APP_ERROR CrowdCount::Inference(const std::vector &inputs, - std::vector &outputs) { - auto dtypes = model_->GetOutputDataType(); - for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) { - std::vector shape = {}; - // 人群计数模型检测特征图尺寸 - for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) { - shape.push_back((uint32_t) modelDesc_.outputTensors[i].tensorDims[j]); - } - // 用检测特征图尺寸分别为数据构建空的tensor - MxBase::TensorBase tensor(shape, dtypes[i], MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor); - if (ret != APP_ERR_OK) { - LogError << "TensorBaseMalloc failed, ret=" << ret << "."; - return ret; - } - outputs.push_back(tensor); - } - // 设置类型为静态batch - MxBase::DynamicInfo dynamicInfo = {}; - dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH; - auto startTime = std::chrono::high_resolution_clock::now(); - APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo); - auto endTime = std::chrono::high_resolution_clock::now(); - double costMs = std::chrono::duration(endTime - startTime).count(); - g_inferCost.push_back(costMs); - if (ret != APP_ERR_OK) { - LogError << "ModelInference failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} -// 后处理 -APP_ERROR CrowdCount::PostProcess(const std::vector &inputs, - std::vector &images, std::vector &results) -{ - // 使用CrowdCountPostProcess post_; - APP_ERROR ret = post_->Process(inputs, images, results); - if (ret != APP_ERR_OK) { - LogError << "Process failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} -// 输出人群计数结果图 -APP_ERROR CrowdCount::WriteResult(const std::vector &outputs, - const std::vector &postimage, const std::vector &results, - const std::string &imgPath) { - cv::Mat mergeImage; - cv::Mat dstimgBgr; - double alpha = 1.0; - double beta = 0.5; - double gamma = 0.0; - cv::Mat imgBgr; - imgBgr = cv::imread(imgPath, cv::IMREAD_COLOR); - imageWidth_ = imgBgr.cols; - imageHeight_ = imgBgr.rows; - int number = results[0]; - std::string str; - str = std::to_string(number); - float fontScale_ = 3; - uint32_t thickness_ = 8; - auto shape = outputs[0].GetShape(); - cv::Mat heatMap = cv::Mat(shape[1], shape[2], CV_8UC3, outputs[0].GetBuffer()); - auto image = postimage[0]; - APP_ERROR ret = image.ToHost(); - if (ret != APP_ERR_OK) { - LogError << GetError(ret) << "ToHost Failed"; - return ret; - } - shape = image.GetShape(); - cv::Mat imageYuv = cv::Mat(shape[0], shape[1], CV_8UC1, image.GetBuffer()); - cv::cvtColor(imageYuv, dstimgBgr, cv::COLOR_YUV2BGR_NV12); - addWeighted(dstimgBgr, alpha, heatMap, beta, gamma, mergeImage); - // 将得到的mergeImage图片的大小还原成被测试图片的大小 - resize(mergeImage, mergeImage, cv::Size(imageWidth_, imageHeight_), 0, 0, cv::INTER_LINEAR); - // 将计算得到的result结果写在输出结果的图片上 - cv::putText(mergeImage, str, cv::Point(30, 120), - cv::FONT_HERSHEY_SIMPLEX, fontScale_, cv::Scalar(0,0,255), thickness_); - cv::imwrite("./result.jpg", mergeImage); - return APP_ERR_OK; -} - -APP_ERROR CrowdCount::Process(const std::string &imgPath) { - // read image - MxBase::TensorBase inTensor; - APP_ERROR ret = ReadImage(imgPath, inTensor); - if (ret != APP_ERR_OK) { - LogError << "ReadImage failed, ret=" << ret << "."; - return ret; - } - //do resize - MxBase::TensorBase outTensor; - ret = Resize(inTensor, outTensor); - if (ret != APP_ERR_OK) { - LogError << "Resize failed, ret=" << ret << "."; - return ret; - } - //do inference - std::vector inputs = {}; - std::vector outputs = {}; - inputs.push_back(outTensor); - ret = Inference(inputs, outputs); - if (ret != APP_ERR_OK) { - LogError << "Inference failed, ret=" << ret << "."; - return ret; - } - LogInfo << "Inference"; - //do postprocess - std::vector heatmap = {}; - std::vector results = {}; - ret = PostProcess(outputs, heatmap, results); - if (ret != APP_ERR_OK) { - LogError << "PostProcess failed, ret=" << ret << "."; - return ret; - } - //wirte result - std::vector postimage = {outTensor}; - ret = WriteResult(heatmap, postimage, results, imgPath); - if (ret != APP_ERR_OK) { - LogError << "Save result failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} -- Gitee From cf5a901227bde94a60ce0e3332759ca14d37a161 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 02:42:51 +0000 Subject: [PATCH 25/27] =?UTF-8?q?=E9=80=82=E9=85=8D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- .../CrowdCounting/CrowdCount/CrowdCount.cpp | 270 ++++++++++++++++++ 1 file changed, 270 insertions(+) create mode 100644 contrib/CrowdCounting/CrowdCount/CrowdCount.cpp diff --git a/contrib/CrowdCounting/CrowdCount/CrowdCount.cpp b/contrib/CrowdCounting/CrowdCount/CrowdCount.cpp new file mode 100644 index 000000000..492f351a5 --- /dev/null +++ b/contrib/CrowdCounting/CrowdCount/CrowdCount.cpp @@ -0,0 +1,270 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "opencv2/opencv.hpp" +#include "CrowdCount.h" +#include "MxBase/DeviceManager/DeviceManager.h" +#include "MxBase/Log/Log.h" +#include +#include +#include + +namespace{ + const uint32_t YUV_BYTE_NU = 3; + const uint32_t YUV_BYTE_DE = 2; + const uint32_t VPC_H_ALIGN = 2; + const uint32_t RESIZE_WIDTH = 1408; + const uint32_t RESIZE_HEIGHT = 800; +} +// 设置配置参数 +void CrowdCount::SetCrowdCountPostProcessConfig(const InitParam &initParam, + std::map &config) { + MxBase::ConfigData configData; + const std::string checkTensor = initParam.checkTensor ? "true" : "false"; + configData.SetJsonValue("CHECK_MODEL", checkTensor); + configData.SetJsonValue("CLASS_NUM", std::to_string(initParam.classNum)); + #ifdef MX_VERSION_5 + auto jsonStr = configData.GetCfgJson().serialize(); + config["postProcessConfigContent"] = jsonStr; + #else + auto jsonStr = configData.GetCfgJson(); + config["postProcessConfigContent"] = jsonStr; + #endif +} + +APP_ERROR CrowdCount::Init(const InitParam &initParam) { + deviceId_ = initParam.deviceId; + APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices(); + if (ret != APP_ERR_OK) { + LogError << "Init devices failed, ret=" << ret << "."; + return ret; + } + ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId); + if (ret != APP_ERR_OK) { + LogError << "Set context failed, ret=" << ret << "."; + return ret; + } + dvppWrapper_ = std::make_shared(); + ret = dvppWrapper_->Init(); + if (ret != APP_ERR_OK) { + LogError << "DvppWrapper init failed, ret=" << ret << "."; + return ret; + } + model_ = std::make_shared(); + ret = model_->Init(initParam.modelPath, modelDesc_); + if (ret != APP_ERR_OK) { + LogError << "ModelInferenceProcessor init failed, ret=" << ret << "."; + return ret; + } + std::map config; + SetCrowdCountPostProcessConfig(initParam, config); + // 初始化CrowdCount后处理对象 + post_ = std::make_shared(); + ret = post_->Init(config); + if (ret != APP_ERR_OK) { + LogError << "CrowdCountstProcess init failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR CrowdCount::DeInit() { + dvppWrapper_->DeInit(); + model_->DeInit(); + post_->DeInit(); + MxBase::DeviceManager::GetInstance()->DestroyDevices(); + return APP_ERR_OK; +} +// 调用mxBase::DvppWrappe.DvppJpegDecode()函数完成图像解码，VpcResize()完成缩放 +APP_ERROR CrowdCount::ReadImage(const std::string &imgPath, MxBase::TensorBase &tensor) { + MxBase::DvppDataInfo output = {}; + APP_ERROR ret = dvppWrapper_->DvppJpegDecode(imgPath, output); + if (ret != APP_ERR_OK) { + LogError << "DvppWrapper DvppJpegDecode failed, ret=" << ret << "."; + return ret; + } + // 将数据转为到DEVICE侧，以便后续处理 + MxBase::MemoryData memoryData((void*)output.data, output.dataSize, + MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + // 对解码后图像对齐尺寸进行判定 + if (output.heightStride % VPC_H_ALIGN != 0) { + LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; + MxBase::MemoryHelper::MxbsFree(memoryData); + return APP_ERR_COMM_INVALID_PARAM; + } + std::vector shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; + tensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); + return APP_ERR_OK; +} + +APP_ERROR CrowdCount::Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor){ + auto shape = inputTensor.GetShape(); + // 还原为原始尺寸 + MxBase::DvppDataInfo input = {}; + input.height = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; + input.width = shape[1]; + input.heightStride = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; + input.widthStride = shape[1]; + input.dataSize = inputTensor.GetByteSize(); + input.data = (uint8_t*)inputTensor.GetBuffer(); + MxBase::ResizeConfig resize = {}; + resize.height = RESIZE_HEIGHT; + resize.width = RESIZE_WIDTH; + MxBase::DvppDataInfo output = {}; + // 图像缩放 + APP_ERROR ret = dvppWrapper_->VpcResize(input, output, resize); + if (ret != APP_ERR_OK) { + LogError << "VpcResize failed, ret=" << ret << "."; + return ret; + } + // 对缩放后图像对齐尺寸进行判定 + MxBase::MemoryData memoryData((void*)output.data, output.dataSize, + MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + if (output.heightStride % VPC_H_ALIGN != 0) { + LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; + MxBase::MemoryHelper::MxbsFree(memoryData); + return APP_ERR_COMM_INVALID_PARAM; + } + shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; + outputTensor = MxBase::TensorBase(memoryData, false, shape,MxBase::TENSOR_DTYPE_UINT8); + return APP_ERR_OK; + +} +// 模型推理 +APP_ERROR CrowdCount::Inference(const std::vector &inputs, + std::vector &outputs) { + auto dtypes = model_->GetOutputDataType(); + for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) { + std::vector shape = {}; + // 人群计数模型检测特征图尺寸 + for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) { + shape.push_back((uint32_t) modelDesc_.outputTensors[i].tensorDims[j]); + } + // 用检测特征图尺寸分别为数据构建空的tensor + MxBase::TensorBase tensor(shape, dtypes[i], MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor); + if (ret != APP_ERR_OK) { + LogError << "TensorBaseMalloc failed, ret=" << ret << "."; + return ret; + } + outputs.push_back(tensor); + } + // 设置类型为静态batch + MxBase::DynamicInfo dynamicInfo = {}; + dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH; + auto startTime = std::chrono::high_resolution_clock::now(); + APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo); + auto endTime = std::chrono::high_resolution_clock::now(); + double costMs = std::chrono::duration(endTime - startTime).count(); + g_inferCost.push_back(costMs); + if (ret != APP_ERR_OK) { + LogError << "ModelInference failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} +// 后处理 +APP_ERROR CrowdCount::PostProcess(const std::vector &inputs, + std::vector &images, std::vector &results) +{ + // 使用CrowdCountPostProcess post_; + APP_ERROR ret = post_->Process(inputs, images, results); + if (ret != APP_ERR_OK) { + LogError << "Process failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} +// 输出人群计数结果图 +APP_ERROR CrowdCount::WriteResult(const std::vector &outputs, + const std::vector &postimage, const std::vector &results, + const std::string &imgPath) { + cv::Mat mergeImage; + cv::Mat dstimgBgr; + double alpha = 1.0; + double beta = 0.5; + double gamma = 0.0; + cv::Mat imgBgr; + imgBgr = cv::imread(imgPath, cv::IMREAD_COLOR); + imageWidth_ = imgBgr.cols; + imageHeight_ = imgBgr.rows; + int number = results[0]; + std::string str; + str = std::to_string(number); + float fontScale_ = 3; + uint32_t thickness_ = 8; + auto shape = outputs[0].GetShape(); + cv::Mat heatMap = cv::Mat(shape[1], shape[2], CV_8UC3, outputs[0].GetBuffer()); + auto image = postimage[0]; + APP_ERROR ret = image.ToHost(); + if (ret != APP_ERR_OK) { + LogError << GetError(ret) << "ToHost Failed"; + return ret; + } + shape = image.GetShape(); + cv::Mat imageYuv = cv::Mat(shape[0], shape[1], CV_8UC1, image.GetBuffer()); + cv::cvtColor(imageYuv, dstimgBgr, cv::COLOR_YUV2BGR_NV12); + addWeighted(dstimgBgr, alpha, heatMap, beta, gamma, mergeImage); + // 将得到的mergeImage图片的大小还原成被测试图片的大小 + resize(mergeImage, mergeImage, cv::Size(imageWidth_, imageHeight_), 0, 0, cv::INTER_LINEAR); + // 将计算得到的result结果写在输出结果的图片上 + cv::putText(mergeImage, str, cv::Point(30, 120), + cv::FONT_HERSHEY_SIMPLEX, fontScale_, cv::Scalar(0,0,255), thickness_); + cv::imwrite("./result.jpg", mergeImage); + return APP_ERR_OK; +} + +APP_ERROR CrowdCount::Process(const std::string &imgPath) { + // read image + MxBase::TensorBase inTensor; + APP_ERROR ret = ReadImage(imgPath, inTensor); + if (ret != APP_ERR_OK) { + LogError << "ReadImage failed, ret=" << ret << "."; + return ret; + } + //do resize + MxBase::TensorBase outTensor; + ret = Resize(inTensor, outTensor); + if (ret != APP_ERR_OK) { + LogError << "Resize failed, ret=" << ret << "."; + return ret; + } + //do inference + std::vector inputs = {}; + std::vector outputs = {}; + inputs.push_back(outTensor); + ret = Inference(inputs, outputs); + if (ret != APP_ERR_OK) { + LogError << "Inference failed, ret=" << ret << "."; + return ret; + } + LogInfo << "Inference"; + //do postprocess + std::vector heatmap = {}; + std::vector results = {}; + ret = PostProcess(outputs, heatmap, results); + if (ret != APP_ERR_OK) { + LogError << "PostProcess failed, ret=" << ret << "."; + return ret; + } + //wirte result + std::vector postimage = {outTensor}; + ret = WriteResult(heatmap, postimage, results, imgPath); + if (ret != APP_ERR_OK) { + LogError << "Save result failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} -- Gitee From 9672a14fd693e934ba865f1a912315b44bbd3b06 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 03:13:21 +0000 Subject: [PATCH 26/27] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20tu?= =?UTF-8?q?torials/DvppWrapperSample/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- tutorials/DvppWrapperSample/README.md | 35 --------------------------- 1 file changed, 35 deletions(-) delete mode 100644 tutorials/DvppWrapperSample/README.md diff --git a/tutorials/DvppWrapperSample/README.md b/tutorials/DvppWrapperSample/README.md deleted file mode 100644 index ce2862963..000000000 --- a/tutorials/DvppWrapperSample/README.md +++ /dev/null @@ -1,35 +0,0 @@ - -# DvppWrapper接口样例 - -## 介绍 -提供DvppWrapper接口样例，对图片实现编码，解码，缩放，抠图，以及把样例图片编码为264视频文件。 - -## 准备 -打开百度图片https://image.baidu.com/，输入任何关键字，然后搜索，右击任意图片，点击另存为。把图片保存在DvppWrapperSample目录下。 - -## 编译与运行 -**步骤1** 修改CMakeLists.txt文件 将set(MX_SDK_HOME ${SDK安装路径}) 中的${SDK安装路径}替换为实际的SDK安装路径 - -**步骤2** 设置环境变量 -ASCEND_HOME Ascend安装的路径，一般为/usr/local/Ascend -LD_LIBRARY_PATH 指定程序运行时依赖的动态库查找路径 -``` -export ASCEND_HOME=/usr/local/Ascend -export ASCEND_VERSION=nnrt/latest -export ARCH_PATTERN=. -export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib/modelpostprocessors:${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/driver/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:${LD_LIBRARY_PATH} -``` - -**步骤3** cd到DvppWrapperSample目录下，执行如下编译命令： -``` -mkdir build -cd build -cmake .. -make -``` - -**步骤4** cd到DvppWrapperSample目录下，可看到可执行文件DvppWrapperSample， 实行命令： -``` -./DvppWrapperSample ./保存的图片 -``` -最后会生成缩放resize_result.jpg、抠图后保存的图片write_result_crop.jpg，以及编码保存的视频test.h264。 \ No newline at end of file -- Gitee From 88f26d7f53e5285ac6cb00ecc81ee44e4c4854fa Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 03:13:40 +0000 Subject: [PATCH 27/27] =?UTF-8?q?=E6=9B=B4=E6=96=B0README?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- tutorials/DvppWrapperSample/README.md | 34 +++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 tutorials/DvppWrapperSample/README.md diff --git a/tutorials/DvppWrapperSample/README.md b/tutorials/DvppWrapperSample/README.md new file mode 100644 index 000000000..f2a780367 --- /dev/null +++ b/tutorials/DvppWrapperSample/README.md @@ -0,0 +1,34 @@ + +# DvppWrapper接口样例 + +## 1 介绍 +提供DvppWrapper接口样例，对图片实现编码，解码，缩放，抠图，以及把样例图片编码为264视频文件。 + +## 2 设置环境变量 +ASCEND_HOME Ascend安装的路径，一般为/usr/local/Ascend +LD_LIBRARY_PATH 指定程序运行时依赖的动态库查找路径 +``` +export ASCEND_HOME=/usr/local/Ascend +export ASCEND_VERSION=nnrt/latest +export ARCH_PATTERN=. +export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib/modelpostprocessors:${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/driver/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:${LD_LIBRARY_PATH} +``` + +## 3 准备模型 +打开百度图片https://image.baidu.com/，输入任何关键字，然后搜索，右击任意图片，点击另存为。把图片保存在DvppWrapperSample目录下。 + +## 4 编译与运行 +**步骤1** 修改CMakeLists.txt文件 将set(MX_SDK_HOME ${SDK安装路径}) 中的${SDK安装路径}替换为实际的SDK安装路径 + +**步骤2** cd到DvppWrapperSample目录下，执行如下编译命令： +``` +mkdir build +cd build +cmake .. +make +``` +**步骤3** cd到DvppWrapperSample目录下，可看到可执行文件DvppWrapperSample， 实行命令： +``` +./DvppWrapperSample ./保存的图片 +``` +最后会生成缩放resize_result.jpg、抠图后保存的图片write_result_crop.jpg，以及编码保存的视频test.h264。 \ No newline at end of file -- Gitee