From 4e3f62e0f59fae7845915bcfe2ea1eb5fc8ca74e Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Fri, 11 Oct 2024 07:05:17 +0000 Subject: [PATCH 01/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/BertTextClassification/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/BertTextClassification/README.md | 222 ----------------------- 1 file changed, 222 deletions(-) delete mode 100644 contrib/BertTextClassification/README.md diff --git a/contrib/BertTextClassification/README.md b/contrib/BertTextClassification/README.md deleted file mode 100644 index 877a73156..000000000 --- a/contrib/BertTextClassification/README.md +++ /dev/null @@ -1,222 +0,0 @@ -# 文本分类 - -## 1. 介绍 - -文本分类插件基于 MindXSDK 开发，在晟腾芯片上进行文本分类，将分类结果保存。输入一段新闻，可以判断该新闻属于哪个类别。 -该模型支持5个新闻类别：体育、健康、军事、教育、汽车。 - -### 1.1 支持的产品 - -本项目以昇腾Atlas310卡为主要的硬件平台。 - -### 1.2 支持的版本 - -支持的SDK版本为2.0.4。 -支持的CANN版本为5.0.4。 - -### 1.3 软件方案介绍 - -基于MindX SDK的文本分类业务流程为：待分类文本通过预处理，将文本根据字典vocab.txt进行编码，组成numpy形式的向量，将向量通过 appsrc 插件输入，然后由模型推理插件mxpi_tensorinfer得到每种类别的得分，再通过后处理插件mxpi_classpostprocessor将模型输出的结果处理，最后得到该文本的类别。本系统的各模块及功能描述如表1.1所示： - - -表1.1 系统方案各子系统功能描述： - -| 序号 | 子系统 | 功能描述 | -| ---- | ------ | ------------ | -| 1 | 文本输入 | 读取输入文本 | -| 2 | 文本编码 | 根据字典对输入文本编码 | -| 3 | 模型推理 | 对文本编码后的张量进行推理 | -| 5 | 后处理 | 从模型推理结果中寻找对应的分类标签 | -| 7 | 保存结果 | 将分类结果保存文件| - -### 1.4 代码目录结构与说明 - -本工程名称为文本分类，工程目录如下图所示： - -``` -. -│ build.sh -│ README.md -│ tree.txt -│ -├─mxBase -│ │ build.sh -│ │ CMakeLists.txt -│ │ main.cpp -│ │ -│ ├─BertClassification -│ │ BertClassification.cpp -│ │ BertClassification.h -│ │ -│ ├─data -│ │ vocab.txt -│ │ -│ ├─model -│ │ bert_text_classification_labels.names -│ │ -│ ├─out -│ │ prediction_label.txt -│ │ -│ └─test -│ Test.cpp -│ Test.h -│ -└─sdk - │ build.sh - │ flowChart.png - │ main.py - │ run.sh - │ tokenizer.py - │ - ├─config - │ bert_text_classification_aipp_tf.cfg - │ bert_text_classification_labels.names - │ - ├─data - │ vocab.txt - │ - ├─model - │ bert_text_classification_aipp_tf.cfg - │ bert_text_classification_labels.names - │ model_conversion.sh - │ - ├─out - │ prediction_label.txt - │ - ├─pipeline - │ BertTextClassification.pipeline - │ - └─test - test.py - test.sh - test_input.py -``` -### 1.5 技术实现流程图 - -![image](sdk/flowChart.png) - - -## 2 环境依赖 - -推荐系统为ubuntu 18.04，环境依赖软件和版本如下表： - -| 软件名称 | 版本 | -| -------- | ------ | -| cmake | 3.10.2 | -| mxVision | 2.0.4 | -| python | 3.9.2 | - -确保环境中正确安装mxVision SDK。 - -在编译运行项目前，需要设置环境变量： - -``` -export MX_SDK_HOME=${SDK安装路径}/mxVision -export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:/usr/local/Ascend/driver/lib64:${LD_LIBRARY_PATH} -export PYTHONPATH=${MX_SDK_HOME}/python:${PYTHONPATH} - -export install_path=/usr/local/Ascend/ascend-toolkit/latest -export PATH=/usr/local/python3.9.2/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH -export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH -export ASCEND_OPP_PATH=${install_path}/opp -``` - -- 环境变量介绍 - -``` -MX_SDK_HOME：MindX SDK mxVision的根安装路径，用于包含MindX SDK提供的所有库和头文件。 -LD_LIBRARY_PATH：提供了MindX SDK已开发的插件和相关的库信息。 -install_path：ascend-toolkit的安装路径。 -PATH：添加python的执行路径和atc转换工具的执行路径。 -LD_LIBRARY_PATH：添加ascend-toolkit和MindX SDK提供的库目录路径。 -ASCEND_OPP_PATH：atc转换工具需要的目录。 -``` - -## 3 模型转换 - -**步骤1** 请参考https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/bert_text_classification.pb -下载模型的pb文件，存放到开发环境普通用户下的任意目录，例如：$HOME/models/bert_text_classification。 - -**步骤2** 执行以下命令使用atc命令进行模型转换： - -cd $HOME/models/bert_text_classification - -atc --model=bert_text_classification.pb --framework=3 --input_format="ND" --output=bert_text_classification --input_shape="input_1:1,300;input_2:1,300" --out_nodes=dense_1/Softmax:0 --soc_version=Ascend310 --op_select_implmode="high_precision" - -**步骤3** 执行以下命令将转换好的模型复制到项目中model文件夹中： - -``` -cp ./bert_text_classification.om $HOME/sdk/model/ -cp ./bert_text_classification.om $HOME/mxBase/model/ -``` - -**步骤4** 执行成功后终端输出为： - -``` -ATC start working now, please wait for a moment. -ATC run success, welcome to the next use. -``` - -## 4 编译与运行 - -**步骤1** 从https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip下载测试数据并解压，解压后的sample.txt和test.csv文件放在项目的mxBase/data和sdk/data目录下。 - -**步骤2** 按照第 2 小节 环境依赖 中的步骤设置环境变量。 - -**步骤3** 按照第 3 小节 模型转换 中的步骤获得 om 模型文件。 - -**步骤4** 将本项目代码的文件路径中出现的 ${SDK目录} 替换成自己SDK的存放目录，下面是需要替换的代码。 - -``` -mxBase目录下的CMakeList.txt中的第13行代码： -set(MX_SDK_HOME ${SDK目录}) - -sdk/pipeline目录下BertTextClassification.pipeline文件中的第26行： -"postProcessLibPath": "${SDK目录}/lib/modelpostprocessors/libresnet50postprocess.so" -``` - -**步骤5** pipeline项目运行在sdk目录下执行命令： - -``` -python3 main.py -``` - -命令执行成功后在out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 - -**步骤6** mxBase项目在mxBase目录中，执行以下代码进行编译。 - -``` -mkdir build -cd build -cmake .. -make -``` - -编译完成后，将可执行文件 mxBase_text_classification 移动到mxBase目录下，执行下面代码运行 - -``` -./mxBase_text_classification ./data/sample.txt -``` - -执行成功后在服务器的mxBase/out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 - -## 5 精度测试 - -**步骤1** 已按照上一小节 编译与运行 的步骤将样例运行成功。 - -**步骤2** 下载[数据集](https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip)后解压，将解压后的test.csv文件分别放在sdk/data目录和mxBase/data目录。 - -**步骤3** pipeline项目中的精度测试文件为sdk/test目录下的test.py，将test.py移到sdk目录下，执行下面代码，得到pipeline的精度测试结果。 - -``` -python3 test.py -``` - -**步骤4** mxBase项目中，将mxBase目录下main.cpp中main方法的全部代码注释，替换为下面代码后执行（即main函数中仅包含以下代码），得到mxBase的精度测试结果。 - -``` -Test::test_accuracy(); -``` - -## 6 其他问题 -1.本项目的设计为限制输入样例为txt文件，其他文件如图片、音频输入则会报错。 \ No newline at end of file -- Gitee From 5eb59c934835cf448ab3a8aed13fd1ea37f1fe4d Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Fri, 11 Oct 2024 07:05:49 +0000 Subject: [PATCH 02/21] =?UTF-8?q?=E6=9B=B4=E6=96=B0README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/BertTextClassification/README.md | 223 +++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 contrib/BertTextClassification/README.md diff --git a/contrib/BertTextClassification/README.md b/contrib/BertTextClassification/README.md new file mode 100644 index 000000000..13562df60 --- /dev/null +++ b/contrib/BertTextClassification/README.md @@ -0,0 +1,223 @@ +# 文本分类 + +## 1. 介绍 + +### 1.1 简介 +文本分类插件基于 MindXSDK 开发，在晟腾芯片上进行文本分类，将分类结果保存。输入一段新闻，可以判断该新闻属于哪个类别。 +该模型支持5个新闻类别：体育、健康、军事、教育、汽车。 + +### 1.2 支持的产品 + +本项目以昇腾Atlas310卡为主要的硬件平台。 + +### 1.3 支持的版本 + +推荐系统为ubuntu 18.04。 + +表1.1 环境依赖软件和版本： + +| 软件名称 | 版本 | +| -------- | ------ | +| cmake | 3.10.2 | +| mxVision | 2.0.4 | +| python | 3.9.2 | +| CANN | 5.0.4 | + +### 1.4 软件方案介绍 + +基于MindX SDK的文本分类业务流程为：待分类文本通过预处理，将文本根据字典vocab.txt进行编码，组成numpy形式的向量，将向量通过 appsrc 插件输入，然后由模型推理插件mxpi_tensorinfer得到每种类别的得分，再通过后处理插件mxpi_classpostprocessor将模型输出的结果处理，最后得到该文本的类别。本系统的各模块及功能描述如表1.2所示： + + +表1.2 系统方案各子系统功能描述： + +| 序号 | 子系统 | 功能描述 | +| ---- | ------ | ------------ | +| 1 | 文本输入 | 读取输入文本 | +| 2 | 文本编码 | 根据字典对输入文本编码 | +| 3 | 模型推理 | 对文本编码后的张量进行推理 | +| 4 | 后处理 | 从模型推理结果中寻找对应的分类标签 | +| 5 | 保存结果 | 将分类结果保存文件| + +### 1.5 代码目录结构与说明 + +本工程名称为文本分类，工程目录如下图所示： + +``` +. +│ build.sh +│ README.md +│ tree.txt +│ +├─mxBase +│ │ build.sh +│ │ CMakeLists.txt +│ │ main.cpp +│ │ +│ ├─BertClassification +│ │ BertClassification.cpp +│ │ BertClassification.h +│ │ +│ ├─data +│ │ vocab.txt +│ │ +│ ├─model +│ │ bert_text_classification_labels.names +│ │ +│ ├─out +│ │ prediction_label.txt +│ │ +│ └─test +│ Test.cpp +│ Test.h +│ +└─sdk + │ build.sh + │ flowChart.png + │ main.py + │ run.sh + │ tokenizer.py + │ + ├─config + │ bert_text_classification_aipp_tf.cfg + │ bert_text_classification_labels.names + │ + ├─data + │ vocab.txt + │ + ├─model + │ bert_text_classification_aipp_tf.cfg + │ bert_text_classification_labels.names + │ model_conversion.sh + │ + ├─out + │ prediction_label.txt + │ + ├─pipeline + │ BertTextClassification.pipeline + │ + └─test + test.py + test.sh + test_input.py +``` +### 1.5 技术实现流程图 + +![image](sdk/flowChart.png) + + +## 2 设置环境变量 + +确保环境中正确安装mxVision SDK。 + +在编译运行项目前，需要设置环境变量： + +``` +export MX_SDK_HOME=${SDK安装路径}/mxVision +export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:/usr/local/Ascend/driver/lib64:${LD_LIBRARY_PATH} +export PYTHONPATH=${MX_SDK_HOME}/python:${PYTHONPATH} + +export install_path=/usr/local/Ascend/ascend-toolkit/latest +export PATH=/usr/local/python3.9.2/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH +export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH +export ASCEND_OPP_PATH=${install_path}/opp +``` + +- 环境变量介绍 + +``` +MX_SDK_HOME：MindX SDK mxVision的根安装路径，用于包含MindX SDK提供的所有库和头文件。 +LD_LIBRARY_PATH：提供了MindX SDK已开发的插件和相关的库信息。 +install_path：ascend-toolkit的安装路径。 +PATH：添加python的执行路径和atc转换工具的执行路径。 +LD_LIBRARY_PATH：添加ascend-toolkit和MindX SDK提供的库目录路径。 +ASCEND_OPP_PATH：atc转换工具需要的目录。 +``` + +## 3 准备模型 + +**步骤1** 请参考https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/bert_text_classification.pb +下载模型的pb文件，存放到开发环境普通用户下的任意目录，例如：$HOME/models/bert_text_classification。 + +**步骤2** 执行以下命令使用atc命令进行模型转换： + +cd $HOME/models/bert_text_classification + +atc --model=bert_text_classification.pb --framework=3 --input_format="ND" --output=bert_text_classification --input_shape="input_1:1,300;input_2:1,300" --out_nodes=dense_1/Softmax:0 --soc_version=Ascend310 --op_select_implmode="high_precision" + +**步骤3** 执行以下命令将转换好的模型复制到项目中model文件夹中： + +``` +cp ./bert_text_classification.om $HOME/sdk/model/ +cp ./bert_text_classification.om $HOME/mxBase/model/ +``` + +**步骤4** 执行成功后终端输出为： + +``` +ATC start working now, please wait for a moment. +ATC run success, welcome to the next use. +``` + +## 4 编译与运行 + +**步骤1** 从https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip下载测试数据并解压，解压后的sample.txt和test.csv文件放在项目的mxBase/data和sdk/data目录下。 + +**步骤2** 按照第 2 小节 环境依赖 中的步骤设置环境变量。 + +**步骤3** 按照第 3 小节 模型转换 中的步骤获得 om 模型文件。 + +**步骤4** 将本项目代码的文件路径中出现的 ${SDK目录} 替换成自己SDK的存放目录，下面是需要替换的代码。 + +``` +mxBase目录下的CMakeList.txt中的第13行代码： +set(MX_SDK_HOME ${SDK目录}) + +sdk/pipeline目录下BertTextClassification.pipeline文件中的第26行： +"postProcessLibPath": "${SDK目录}/lib/modelpostprocessors/libresnet50postprocess.so" +``` + +**步骤5** pipeline项目运行在sdk目录下执行命令： + +``` +python3 main.py +``` + +命令执行成功后在out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 + +**步骤6** mxBase项目在mxBase目录中，执行以下代码进行编译。 + +``` +mkdir build +cd build +cmake .. +make +``` + +编译完成后，将可执行文件 mxBase_text_classification 移动到mxBase目录下，执行下面代码运行 + +``` +./mxBase_text_classification ./data/sample.txt +``` + +执行成功后在服务器的mxBase/out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 + +## 5 精度测试 + +**步骤1** 已按照上一小节 编译与运行 的步骤将样例运行成功。 + +**步骤2** 下载[数据集](https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip)后解压，将解压后的test.csv文件分别放在sdk/data目录和mxBase/data目录。 + +**步骤3** pipeline项目中的精度测试文件为sdk/test目录下的test.py，将test.py移到sdk目录下，执行下面代码，得到pipeline的精度测试结果。 + +``` +python3 test.py +``` + +**步骤4** mxBase项目中，将mxBase目录下main.cpp中main方法的全部代码注释，替换为下面代码（即main函数中仅包含以下代码），然后重新编译并运行，得到mxBase的精度测试结果。 + +``` +Test::test_accuracy(); +``` + +## 6 其他问题 +1.本项目的设计为限制输入样例为txt文件，其他文件如图片、音频输入则会报错。 \ No newline at end of file -- Gitee From aa170cfafb240d55d5f71ae01ba00fd9275089b3 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Fri, 11 Oct 2024 07:09:15 +0000 Subject: [PATCH 03/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/BertTextClassification/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/BertTextClassification/README.md | 223 ----------------------- 1 file changed, 223 deletions(-) delete mode 100644 contrib/BertTextClassification/README.md diff --git a/contrib/BertTextClassification/README.md b/contrib/BertTextClassification/README.md deleted file mode 100644 index 13562df60..000000000 --- a/contrib/BertTextClassification/README.md +++ /dev/null @@ -1,223 +0,0 @@ -# 文本分类 - -## 1. 介绍 - -### 1.1 简介 -文本分类插件基于 MindXSDK 开发，在晟腾芯片上进行文本分类，将分类结果保存。输入一段新闻，可以判断该新闻属于哪个类别。 -该模型支持5个新闻类别：体育、健康、军事、教育、汽车。 - -### 1.2 支持的产品 - -本项目以昇腾Atlas310卡为主要的硬件平台。 - -### 1.3 支持的版本 - -推荐系统为ubuntu 18.04。 - -表1.1 环境依赖软件和版本： - -| 软件名称 | 版本 | -| -------- | ------ | -| cmake | 3.10.2 | -| mxVision | 2.0.4 | -| python | 3.9.2 | -| CANN | 5.0.4 | - -### 1.4 软件方案介绍 - -基于MindX SDK的文本分类业务流程为：待分类文本通过预处理，将文本根据字典vocab.txt进行编码，组成numpy形式的向量，将向量通过 appsrc 插件输入，然后由模型推理插件mxpi_tensorinfer得到每种类别的得分，再通过后处理插件mxpi_classpostprocessor将模型输出的结果处理，最后得到该文本的类别。本系统的各模块及功能描述如表1.2所示： - - -表1.2 系统方案各子系统功能描述： - -| 序号 | 子系统 | 功能描述 | -| ---- | ------ | ------------ | -| 1 | 文本输入 | 读取输入文本 | -| 2 | 文本编码 | 根据字典对输入文本编码 | -| 3 | 模型推理 | 对文本编码后的张量进行推理 | -| 4 | 后处理 | 从模型推理结果中寻找对应的分类标签 | -| 5 | 保存结果 | 将分类结果保存文件| - -### 1.5 代码目录结构与说明 - -本工程名称为文本分类，工程目录如下图所示： - -``` -. -│ build.sh -│ README.md -│ tree.txt -│ -├─mxBase -│ │ build.sh -│ │ CMakeLists.txt -│ │ main.cpp -│ │ -│ ├─BertClassification -│ │ BertClassification.cpp -│ │ BertClassification.h -│ │ -│ ├─data -│ │ vocab.txt -│ │ -│ ├─model -│ │ bert_text_classification_labels.names -│ │ -│ ├─out -│ │ prediction_label.txt -│ │ -│ └─test -│ Test.cpp -│ Test.h -│ -└─sdk - │ build.sh - │ flowChart.png - │ main.py - │ run.sh - │ tokenizer.py - │ - ├─config - │ bert_text_classification_aipp_tf.cfg - │ bert_text_classification_labels.names - │ - ├─data - │ vocab.txt - │ - ├─model - │ bert_text_classification_aipp_tf.cfg - │ bert_text_classification_labels.names - │ model_conversion.sh - │ - ├─out - │ prediction_label.txt - │ - ├─pipeline - │ BertTextClassification.pipeline - │ - └─test - test.py - test.sh - test_input.py -``` -### 1.5 技术实现流程图 - -![image](sdk/flowChart.png) - - -## 2 设置环境变量 - -确保环境中正确安装mxVision SDK。 - -在编译运行项目前，需要设置环境变量： - -``` -export MX_SDK_HOME=${SDK安装路径}/mxVision -export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:/usr/local/Ascend/driver/lib64:${LD_LIBRARY_PATH} -export PYTHONPATH=${MX_SDK_HOME}/python:${PYTHONPATH} - -export install_path=/usr/local/Ascend/ascend-toolkit/latest -export PATH=/usr/local/python3.9.2/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH -export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH -export ASCEND_OPP_PATH=${install_path}/opp -``` - -- 环境变量介绍 - -``` -MX_SDK_HOME：MindX SDK mxVision的根安装路径，用于包含MindX SDK提供的所有库和头文件。 -LD_LIBRARY_PATH：提供了MindX SDK已开发的插件和相关的库信息。 -install_path：ascend-toolkit的安装路径。 -PATH：添加python的执行路径和atc转换工具的执行路径。 -LD_LIBRARY_PATH：添加ascend-toolkit和MindX SDK提供的库目录路径。 -ASCEND_OPP_PATH：atc转换工具需要的目录。 -``` - -## 3 准备模型 - -**步骤1** 请参考https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/bert_text_classification.pb -下载模型的pb文件，存放到开发环境普通用户下的任意目录，例如：$HOME/models/bert_text_classification。 - -**步骤2** 执行以下命令使用atc命令进行模型转换： - -cd $HOME/models/bert_text_classification - -atc --model=bert_text_classification.pb --framework=3 --input_format="ND" --output=bert_text_classification --input_shape="input_1:1,300;input_2:1,300" --out_nodes=dense_1/Softmax:0 --soc_version=Ascend310 --op_select_implmode="high_precision" - -**步骤3** 执行以下命令将转换好的模型复制到项目中model文件夹中： - -``` -cp ./bert_text_classification.om $HOME/sdk/model/ -cp ./bert_text_classification.om $HOME/mxBase/model/ -``` - -**步骤4** 执行成功后终端输出为： - -``` -ATC start working now, please wait for a moment. -ATC run success, welcome to the next use. -``` - -## 4 编译与运行 - -**步骤1** 从https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip下载测试数据并解压，解压后的sample.txt和test.csv文件放在项目的mxBase/data和sdk/data目录下。 - -**步骤2** 按照第 2 小节 环境依赖 中的步骤设置环境变量。 - -**步骤3** 按照第 3 小节 模型转换 中的步骤获得 om 模型文件。 - -**步骤4** 将本项目代码的文件路径中出现的 ${SDK目录} 替换成自己SDK的存放目录，下面是需要替换的代码。 - -``` -mxBase目录下的CMakeList.txt中的第13行代码： -set(MX_SDK_HOME ${SDK目录}) - -sdk/pipeline目录下BertTextClassification.pipeline文件中的第26行： -"postProcessLibPath": "${SDK目录}/lib/modelpostprocessors/libresnet50postprocess.so" -``` - -**步骤5** pipeline项目运行在sdk目录下执行命令： - -``` -python3 main.py -``` - -命令执行成功后在out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 - -**步骤6** mxBase项目在mxBase目录中，执行以下代码进行编译。 - -``` -mkdir build -cd build -cmake .. -make -``` - -编译完成后，将可执行文件 mxBase_text_classification 移动到mxBase目录下，执行下面代码运行 - -``` -./mxBase_text_classification ./data/sample.txt -``` - -执行成功后在服务器的mxBase/out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 - -## 5 精度测试 - -**步骤1** 已按照上一小节 编译与运行 的步骤将样例运行成功。 - -**步骤2** 下载[数据集](https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip)后解压，将解压后的test.csv文件分别放在sdk/data目录和mxBase/data目录。 - -**步骤3** pipeline项目中的精度测试文件为sdk/test目录下的test.py，将test.py移到sdk目录下，执行下面代码，得到pipeline的精度测试结果。 - -``` -python3 test.py -``` - -**步骤4** mxBase项目中，将mxBase目录下main.cpp中main方法的全部代码注释，替换为下面代码（即main函数中仅包含以下代码），然后重新编译并运行，得到mxBase的精度测试结果。 - -``` -Test::test_accuracy(); -``` - -## 6 其他问题 -1.本项目的设计为限制输入样例为txt文件，其他文件如图片、音频输入则会报错。 \ No newline at end of file -- Gitee From 30494a121a5eefa1b7b0d5a1489fb69643b204a0 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Fri, 11 Oct 2024 07:10:13 +0000 Subject: [PATCH 04/21] =?UTF-8?q?=E6=9B=B4=E6=96=B0README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/BertTextClassification/README.md | 223 +++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 contrib/BertTextClassification/README.md diff --git a/contrib/BertTextClassification/README.md b/contrib/BertTextClassification/README.md new file mode 100644 index 000000000..119c12544 --- /dev/null +++ b/contrib/BertTextClassification/README.md @@ -0,0 +1,223 @@ +# 文本分类 + +## 1. 介绍 + +### 1.1 简介 +文本分类插件基于 MindXSDK 开发，在晟腾芯片上进行文本分类，将分类结果保存。输入一段新闻，可以判断该新闻属于哪个类别。 +该模型支持5个新闻类别：体育、健康、军事、教育、汽车。 + +### 1.2 支持的产品 + +本项目以昇腾Atlas310卡为主要的硬件平台。 + +### 1.3 支持的版本 + +推荐系统为ubuntu 18.04。 + +表1.1 环境依赖软件和版本： + +| 软件名称 | 版本 | +| -------- | ------ | +| cmake | 3.10.2 | +| mxVision | 2.0.4 | +| python | 3.9.2 | +| CANN | 5.0.4 | + +### 1.4 软件方案介绍 + +基于MindX SDK的文本分类业务流程为：待分类文本通过预处理，将文本根据字典vocab.txt进行编码，组成numpy形式的向量，将向量通过 appsrc 插件输入，然后由模型推理插件mxpi_tensorinfer得到每种类别的得分，再通过后处理插件mxpi_classpostprocessor将模型输出的结果处理，最后得到该文本的类别。本系统的各模块及功能描述如表1.2所示： + + +表1.2 系统方案各子系统功能描述： + +| 序号 | 子系统 | 功能描述 | +| ---- | ------ | ------------ | +| 1 | 文本输入 | 读取输入文本 | +| 2 | 文本编码 | 根据字典对输入文本编码 | +| 3 | 模型推理 | 对文本编码后的张量进行推理 | +| 4 | 后处理 | 从模型推理结果中寻找对应的分类标签 | +| 5 | 保存结果 | 将分类结果保存文件| + +### 1.5 代码目录结构与说明 + +本工程名称为文本分类，工程目录如下图所示： + +``` +. +│ build.sh +│ README.md +│ tree.txt +│ +├─mxBase +│ │ build.sh +│ │ CMakeLists.txt +│ │ main.cpp +│ │ +│ ├─BertClassification +│ │ BertClassification.cpp +│ │ BertClassification.h +│ │ +│ ├─data +│ │ vocab.txt +│ │ +│ ├─model +│ │ bert_text_classification_labels.names +│ │ +│ ├─out +│ │ prediction_label.txt +│ │ +│ └─test +│ Test.cpp +│ Test.h +│ +└─sdk + │ build.sh + │ flowChart.png + │ main.py + │ run.sh + │ tokenizer.py + │ + ├─config + │ bert_text_classification_aipp_tf.cfg + │ bert_text_classification_labels.names + │ + ├─data + │ vocab.txt + │ + ├─model + │ bert_text_classification_aipp_tf.cfg + │ bert_text_classification_labels.names + │ model_conversion.sh + │ + ├─out + │ prediction_label.txt + │ + ├─pipeline + │ BertTextClassification.pipeline + │ + └─test + test.py + test.sh + test_input.py +``` +### 1.6 技术实现流程图 + +![image](sdk/flowChart.png) + + +## 2 设置环境变量 + +确保环境中正确安装mxVision SDK。 + +在编译运行项目前，需要设置环境变量： + +``` +export MX_SDK_HOME=${SDK安装路径}/mxVision +export LD_LIBRARY_PATH=${MX_SDK_HOME}/lib:${MX_SDK_HOME}/opensource/lib:${MX_SDK_HOME}/opensource/lib64:/usr/local/Ascend/ascend-toolkit/latest/acllib/lib64:/usr/local/Ascend/driver/lib64:${LD_LIBRARY_PATH} +export PYTHONPATH=${MX_SDK_HOME}/python:${PYTHONPATH} + +export install_path=/usr/local/Ascend/ascend-toolkit/latest +export PATH=/usr/local/python3.9.2/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH +export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH +export ASCEND_OPP_PATH=${install_path}/opp +``` + +- 环境变量介绍 + +``` +MX_SDK_HOME：MindX SDK mxVision的根安装路径，用于包含MindX SDK提供的所有库和头文件。 +LD_LIBRARY_PATH：提供了MindX SDK已开发的插件和相关的库信息。 +install_path：ascend-toolkit的安装路径。 +PATH：添加python的执行路径和atc转换工具的执行路径。 +LD_LIBRARY_PATH：添加ascend-toolkit和MindX SDK提供的库目录路径。 +ASCEND_OPP_PATH：atc转换工具需要的目录。 +``` + +## 3 准备模型 + +**步骤1** 请参考https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/bert_text_classification.pb +下载模型的pb文件，存放到开发环境普通用户下的任意目录，例如：$HOME/models/bert_text_classification。 + +**步骤2** 执行以下命令使用atc命令进行模型转换： + +cd $HOME/models/bert_text_classification + +atc --model=bert_text_classification.pb --framework=3 --input_format="ND" --output=bert_text_classification --input_shape="input_1:1,300;input_2:1,300" --out_nodes=dense_1/Softmax:0 --soc_version=Ascend310 --op_select_implmode="high_precision" + +**步骤3** 执行以下命令将转换好的模型复制到项目中model文件夹中： + +``` +cp ./bert_text_classification.om $HOME/sdk/model/ +cp ./bert_text_classification.om $HOME/mxBase/model/ +``` + +**步骤4** 执行成功后终端输出为： + +``` +ATC start working now, please wait for a moment. +ATC run success, welcome to the next use. +``` + +## 4 编译与运行 + +**步骤1** 从https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip下载测试数据并解压，解压后的sample.txt和test.csv文件放在项目的mxBase/data和sdk/data目录下。 + +**步骤2** 按照第 2 小节 环境依赖 中的步骤设置环境变量。 + +**步骤3** 按照第 3 小节 模型转换 中的步骤获得 om 模型文件。 + +**步骤4** 将本项目代码的文件路径中出现的 ${SDK目录} 替换成自己SDK的存放目录，下面是需要替换的代码。 + +``` +mxBase目录下的CMakeList.txt中的第13行代码： +set(MX_SDK_HOME ${SDK目录}) + +sdk/pipeline目录下BertTextClassification.pipeline文件中的第26行： +"postProcessLibPath": "${SDK目录}/lib/modelpostprocessors/libresnet50postprocess.so" +``` + +**步骤5** pipeline项目运行在sdk目录下执行命令： + +``` +python3 main.py +``` + +命令执行成功后在out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 + +**步骤6** mxBase项目在mxBase目录中，执行以下代码进行编译。 + +``` +mkdir build +cd build +cmake .. +make +``` + +编译完成后，将可执行文件 mxBase_text_classification 移动到mxBase目录下，执行下面代码运行 + +``` +./mxBase_text_classification ./data/sample.txt +``` + +执行成功后在服务器的mxBase/out目录下生成分类结果文件 prediction_label.txt，查看结果文件验证分类结果。 + +## 5 精度测试 + +**步骤1** 已按照上一小节 编译与运行 的步骤将样例运行成功。 + +**步骤2** 下载[数据集](https://mindx.sdk.obs.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/BertTextClassification/data.zip)后解压，将解压后的test.csv文件分别放在sdk/data目录和mxBase/data目录。 + +**步骤3** pipeline项目中的精度测试文件为sdk/test目录下的test.py，将test.py移到sdk目录下，执行下面代码，得到pipeline的精度测试结果。 + +``` +python3 test.py +``` + +**步骤4** mxBase项目中，将mxBase目录下main.cpp中main方法的全部代码注释，替换为下面代码（即main函数中仅包含以下代码），参考第4小节 编译与运行 中的步骤4重新编译并运行，得到mxBase的精度测试结果。 + +``` +Test::test_accuracy(); +``` + +## 6 常见问题 +1.本项目的设计为限制输入样例为txt文件，其他文件如图片、音频输入则会报错。 \ No newline at end of file -- Gitee From 4d96525ce6ff01f84c12788298128d46a0233f43 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:13:23 +0000 Subject: [PATCH 05/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/CMakeLists.txt?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/CMakeLists.txt | 37 --------------------- 1 file changed, 37 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/CMakeLists.txt diff --git a/contrib/EdgeDetectionPicture/CMakeLists.txt b/contrib/EdgeDetectionPicture/CMakeLists.txt deleted file mode 100644 index 3928c17d7..000000000 --- a/contrib/EdgeDetectionPicture/CMakeLists.txt +++ /dev/null @@ -1,37 +0,0 @@ -cmake_minimum_required(VERSION 3.10) -project(edge_detection_picture) - -include_directories(./rcfPostProcess) -include_directories(./rcfDetection) -file(GLOB_RECURSE RCF_POSTPROCESS ${PROJECT_SOURCE_DIR}/rcfPostProcess/*cpp) -file(GLOB_RECURSE RCF_DETECTION ${PROJECT_SOURCE_DIR}/rcfDetection/*cpp) -set(TARGET edge_detection_picture) -add_compile_options(-std=c++11 -fPIE -fstack-protector-all -fPIC -Wl,-z,relro,-z,now,-z,noexecstack -s -pie -Wall) -add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) - -set(MX_SDK_HOME "$ENV{MX_SDK_HOME}") - -include_directories( - ${MX_SDK_HOME}/include - ${MX_SDK_HOME}/opensource/include - ${MX_SDK_HOME}/opensource/include/opencv4 -) - -link_directories( - ${MX_SDK_HOME}/lib - ${MX_SDK_HOME}/opensource/lib - ${MX_SDK_HOME}/lib/modelpostprocessors - ${MX_SDK_HOME}/include/MxBase/postprocess/include - /usr/local/Ascend/ascend-toolkit/latest/acllib/lib64 - /usr/local/Ascend/driver/lib64/ -) - -add_executable(edge_detection_picture main.cpp ${RCF_DETECTION} ${RCF_POSTPROCESS}) -target_link_libraries(edge_detection_picture - glog - mxbase - opencv_world - boost_system - boost_filesystem - ) -install(TARGETS ${TARGET} RUNTIME DESTINATION ${PROJECT_SOURCE_DIR}/) -- Gitee From 061a83867b246235cdfa347191776a192943782a Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:13:50 +0000 Subject: [PATCH 06/21] =?UTF-8?q?=E6=9B=B4=E6=96=B0CMake=E4=BD=BF=E5=85=B6?= =?UTF-8?q?=E9=80=82=E9=85=8D=E7=89=88=E6=9C=AC?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/EdgeDetectionPicture/CMakeLists.txt | 43 +++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 contrib/EdgeDetectionPicture/CMakeLists.txt diff --git a/contrib/EdgeDetectionPicture/CMakeLists.txt b/contrib/EdgeDetectionPicture/CMakeLists.txt new file mode 100644 index 000000000..c038f4269 --- /dev/null +++ b/contrib/EdgeDetectionPicture/CMakeLists.txt @@ -0,0 +1,43 @@ +cmake_minimum_required(VERSION 3.10) +project(edge_detection_picture) + +include_directories(./rcfPostProcess) +include_directories(./rcfDetection) +file(GLOB_RECURSE RCF_POSTPROCESS ${PROJECT_SOURCE_DIR}/rcfPostProcess/*cpp) +file(GLOB_RECURSE RCF_DETECTION ${PROJECT_SOURCE_DIR}/rcfDetection/*cpp) +set(TARGET edge_detection_picture) +add_compile_options(-std=c++11 -fPIE -fstack-protector-all -fPIC -Wl,-z,relro,-z,now,-z,noexecstack -s -pie -Wall) +add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0 -Dgoogle=mindxsdk_private) + +set(MX_SDK_HOME /root/SDK/mxVision) + +set(cpprest_DIR ${MX_SDK_HOME}/opensource/lib/libcpprest.so) +if(EXISTS ${cpprest_DIR}) + target_link_libraries(edge_detection_picture cpprest) + add_definitions(_DMX_VERSION_5) +endif() + +include_directories( + ${MX_SDK_HOME}/include + ${MX_SDK_HOME}/opensource/include + ${MX_SDK_HOME}/opensource/include/opencv4 +) + +link_directories( + ${MX_SDK_HOME}/lib + ${MX_SDK_HOME}/opensource/lib + ${MX_SDK_HOME}/lib/modelpostprocessors + ${MX_SDK_HOME}/include/MxBase/postprocess/include + /usr/local/Ascend/ascend-toolkit/latest/acllib/lib64 + /usr/local/Ascend/driver/lib64/ +) + +add_executable(edge_detection_picture main.cpp ${RCF_DETECTION} ${RCF_POSTPROCESS}) +target_link_libraries(edge_detection_picture + glog + mxbase + opencv_world + boost_system + boost_filesystem + ) +install(TARGETS ${TARGET} RUNTIME DESTINATION ${PROJECT_SOURCE_DIR}/) -- Gitee From fb9df239a505356210a7f2c62c34abb014b5757e Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:14:00 +0000 Subject: [PATCH 07/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/rcfDetection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../rcfDetection/RcfDetection.cpp | 281 ------------------ .../rcfDetection/RcfDetection.h | 61 ---- 2 files changed, 342 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp delete mode 100644 contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h diff --git a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp deleted file mode 100644 index a8a84460c..000000000 --- a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp +++ /dev/null @@ -1,281 +0,0 @@ -/** - * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#include "RcfDetection.h" -#include "MxBase/DeviceManager/DeviceManager.h" -#include "MxBase/Log/Log.h" -#include "../rcfPostProcess/RcfPostProcess.h" -#include -#include -#include "opencv2/opencv.hpp" - -namespace { - const uint32_t YUV_BYTE_NU = 3; - const uint32_t YUV_BYTE_DE = 2; - const uint32_t VPC_H_ALIGN = 2; - const uint32_t channel = 3; - const double ALPHA = 255.0 / 255; -} -void RcfDetection::SetRcfPostProcessConfig(const InitParam &initParam, - std::map &config) -{ - MxBase::ConfigData configData; - const std::string checkTensor = initParam.checkTensor ? "true" : "false"; - configData.SetJsonValue("OUTSIZE_NUM", std::to_string(initParam.outSizeNum)); - configData.SetJsonValue("OUTSIZE", initParam.outSize); - configData.SetJsonValue("RCF_TYPE", std::to_string(initParam.rcfType)); - configData.SetJsonValue("MODEL_TYPE", std::to_string(initParam.modelType)); - configData.SetJsonValue("INPUT_TYPE", std::to_string(initParam.inputType)); - configData.SetJsonValue("CHECK_MODEL", checkTensor); - auto jsonStr = configData.GetCfgJson().serialize(); - config["postProcessConfigContent"] = jsonStr; -} - -APP_ERROR RcfDetection::Init(const InitParam &initParam) -{ - deviceId_ = initParam.deviceId; - APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices(); - if (ret != APP_ERR_OK) { - LogError << "Init devices failed, ret=" << ret << "."; - return ret; - } - ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId); - if (ret != APP_ERR_OK) { - LogError << "Set context failed, ret=" << ret << "."; - return ret; - } - dvppWrapper_ = std::make_shared(); - ret = dvppWrapper_->Init(); - if (ret != APP_ERR_OK) { - LogError << "DvppWrapper init failed, ret=" << ret << "."; - return ret; - } - model_ = std::make_shared(); - ret = model_->Init(initParam.modelPath, modelDesc_); - if (ret != APP_ERR_OK) { - LogError << "ModelInferenceProcessor init failed, ret=" << ret << "."; - return ret; - } - - std::map config; - SetRcfPostProcessConfig(initParam, config); - post_ = std::make_shared(); - ret = post_->Init(config); - if (ret != APP_ERR_OK) { - LogError << "RcfPostProcess init failed, ret=" << ret << "."; - return ret; - } - if (ret != APP_ERR_OK) { - LogError << "Failed to load labels, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::DeInit() -{ - dvppWrapper_->DeInit(); - model_->DeInit(); - post_->DeInit(); - MxBase::DeviceManager::GetInstance()->DestroyDevices(); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::ReadImage(const std::string &imgPath, - MxBase::TensorBase &tensor) -{ - MxBase::DvppDataInfo output = {}; - APP_ERROR ret = dvppWrapper_->DvppJpegDecode(imgPath, output); - if (ret != APP_ERR_OK) { - LogError << "DvppWrapper DvppJpegDecode failed, ret=" << ret << "."; - return ret; - } - MxBase::MemoryData memoryData((void*)output.data, output.dataSize, - MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - if (output.heightStride % VPC_H_ALIGN != 0) { - LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; - MxBase::MemoryHelper::MxbsFree(memoryData); - return APP_ERR_COMM_INVALID_PARAM; - } - dvppHeightStride = output.heightStride; - dvppWidthStride = output.widthStride; - std::vector shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; - tensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); - return APP_ERR_OK; -} -APP_ERROR RcfDetection::Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, - uint32_t resizeHeight, uint32_t resizeWidth) -{ - auto shape = inputTensor.GetShape(); - MxBase::DvppDataInfo input = {}; - input.height = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; - input.width = shape[1]; - input.heightStride = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; - input.widthStride = shape[1]; - input.dataSize = inputTensor.GetByteSize(); - input.data = (uint8_t*)inputTensor.GetBuffer(); - MxBase::ResizeConfig resize = {}; - resize.height = resizeHeight; - resize.width = resizeWidth; - MxBase::DvppDataInfo output = {}; - APP_ERROR ret = dvppWrapper_->VpcResize(input, output, resize); - if (ret != APP_ERR_OK) { - LogError << "VpcResize failed, ret=" << ret << "."; - return ret; - } - MxBase::MemoryData memoryData((void*)output.data, output.dataSize, - MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - if (output.heightStride % VPC_H_ALIGN != 0) { - LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; - MxBase::MemoryHelper::MxbsFree(memoryData); - return APP_ERR_COMM_INVALID_PARAM; - } - shape = {1, channel, output.heightStride, output.widthStride}; - outputTensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::Inference(const std::vector &inputs, - std::vector &outputs) -{ - std::vector output={}; - auto dtypes = model_->GetOutputDataType(); - for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) { - std::vector shape = {}; - for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) { - shape.push_back((uint32_t)modelDesc_.outputTensors[i].tensorDims[j]); - } - MxBase::TensorBase tensor(shape, dtypes[i], MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor); - if (ret != APP_ERR_OK) { - LogError << "TensorBaseMalloc failed, ret=" << ret << "."; - return ret; - } - outputs.push_back(tensor); - } - MxBase::DynamicInfo dynamicInfo = {}; - dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH; - auto startTime = std::chrono::high_resolution_clock::now(); - APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo); - auto endTime = std::chrono::high_resolution_clock::now(); - double costMs = std::chrono::duration(endTime - startTime).count(); - LogInfo<< "costMs:"<< costMs; - if (ret != APP_ERR_OK) { - LogError << "ModelInference failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::PostProcess(const MxBase::TensorBase &tensor, - const std::vector &outputs, - std::vector &postProcessOutput) -{ - auto shape = tensor.GetShape(); - MxBase::ResizedImageInfo imgInfo; - imgInfo.widthOriginal = shape[1]; - imgInfo.heightOriginal = shape[0] * YUV_BYTE_DE; - uint32_t widthResize = 512; - uint32_t heightResize = 512; - imgInfo.widthResize = widthResize; - imgInfo.heightResize = heightResize; - imgInfo.resizeType = MxBase::RESIZER_STRETCHING; - std::vector imageInfoVec = {}; - imageInfoVec.push_back(imgInfo); - APP_ERROR ret = post_->Process(outputs, postProcessOutput); - if (ret != APP_ERR_OK) { - LogError << "Process failed, ret=" << ret << "."; - return ret; - } - ret = post_->DeInit(); - if (ret != APP_ERR_OK) { - LogError << "RcfDetection DeInit failed"; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath) -{ - auto shape = inferTensor.GetShape(); - int dim2 = 2; - int dim3 = 3; - uint32_t height = shape[dim2]; - uint32_t width = shape[dim3]; - cv::Mat imgBgr = cv::imread(imgPath); - std::string fileName = imgPath.substr(imgPath.find_last_of("/") + 1); - uint32_t imageWidth = imgBgr.cols; - uint32_t imageHeight = imgBgr.rows; - cv::Mat modelOutput = cv::Mat(height, width, CV_32FC1, inferTensor.GetBuffer()); - cv::Mat grayMat; - cv::Mat resizedMat; - int crop = 5; - cv::Rect myROI(0, 0, imageWidth - crop, imageHeight); - resize(modelOutput, resizedMat, cv::Size(dvppWidthStride, dvppHeightStride), 0, 0, cv::INTER_LINEAR); - resizedMat.convertTo(grayMat, CV_8UC1, ALPHA); - cv::Mat croppedImage = grayMat(myROI); - resize(croppedImage, croppedImage, cv::Size(imageWidth, imageHeight), 0, 0, cv::INTER_LINEAR); - - std::string resultPathName = "result"; - if (access(resultPathName.c_str(), 0) != 0) { - int ret = mkdir(resultPathName.c_str(), S_IRUSR | S_IWUSR | S_IXUSR); - if (ret != 0) { - LogError << "Failed to create result directory: " << resultPathName << ", ret = " << ret; - return APP_ERR_COMM_FAILURE; - } - } - - cv::imwrite("./result/"+fileName, croppedImage); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::Process(const std::string &imgPath) -{ - MxBase::TensorBase inTensor; - APP_ERROR ret = ReadImage(imgPath, inTensor); - if (ret != APP_ERR_OK) { - LogError << "ReadImage failed, ret=" << ret << "."; - return ret; - } - MxBase::TensorBase outTensor; - uint32_t resizeHeight = 512; - uint32_t resizeWidth = 512; - ret = Resize(inTensor, outTensor, resizeHeight, resizeWidth); - if (ret != APP_ERR_OK) { - LogError << "Resize failed, ret=" << ret << "."; - return ret; - } - std::vector inputs = {}; - std::vector outputs = {}; - auto shape = outTensor.GetShape(); - inputs.push_back(outTensor); - ret = Inference(inputs, outputs); - if (ret != APP_ERR_OK) { - LogError << "Inference failed, ret=" << ret << "."; - return ret; - } - std::vector postProcessOutput={}; - ret = PostProcess(inTensor, outputs, postProcessOutput); - if (ret != APP_ERR_OK) { - LogError << "PostProcess failed, ret=" << ret << "."; - return ret; - } - ret = WriteResult(postProcessOutput[0], imgPath); - if (ret != APP_ERR_OK) { - LogError << "Save result failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} diff --git a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h deleted file mode 100644 index 7c0ae6da4..000000000 --- a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h +++ /dev/null @@ -1,61 +0,0 @@ -/* - * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#ifndef MXBASE_RCFDETECTION_H -#define MXBASE_RCFDETECTION_H - -#include -#include -#include "MxBase/DvppWrapper/DvppWrapper.h" -#include "MxBase/ModelInfer/ModelInferenceProcessor.h" -#include "MxBase/Tensor/TensorContext/TensorContext.h" - -struct InitParam { - uint32_t deviceId = 0; - bool checkTensor = true; - std::string modelPath = ""; - uint32_t outSizeNum = 0; - std::string outSize = ""; - uint32_t rcfType = 0; - uint32_t modelType = 0; - uint32_t inputType = 0; -}; - -class RcfDetection { -public: - APP_ERROR Init(const InitParam &initParam); - APP_ERROR DeInit(); - APP_ERROR Inference(const std::vector &inputs, std::vector &outputs); - APP_ERROR PostProcess(const MxBase::TensorBase &tensor, const std::vector &outputs, - std::vector &postProcessOutput); - APP_ERROR Process(const std::string &imgPath); -protected: - APP_ERROR ReadImage(const std::string &imgPath, MxBase::TensorBase &tensor); - APP_ERROR Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, - uint32_t resizeHeight, uint32_t resizeWidth); - APP_ERROR WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath); - void SetRcfPostProcessConfig(const InitParam &initParam, std::map &config); - -private: - std::shared_ptr dvppWrapper_; - std::shared_ptr model_; - std::shared_ptr post_; - MxBase::ModelDesc modelDesc_ = {}; - uint32_t deviceId_ = 0; - int dvppHeightStride = 0; - int dvppWidthStride = 0; -}; -#endif -- Gitee From 5a5c1e129e621eb7dd649e47bf5a9a9004c00f3c Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:14:14 +0000 Subject: [PATCH 08/21] =?UTF-8?q?=E6=96=B0=E5=BB=BA=20rcfDetection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/rcfDetection/.keep | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 contrib/EdgeDetectionPicture/rcfDetection/.keep diff --git a/contrib/EdgeDetectionPicture/rcfDetection/.keep b/contrib/EdgeDetectionPicture/rcfDetection/.keep new file mode 100644 index 000000000..e69de29bb -- Gitee From 9eaf6d186dd3a44d9db2e9cac335c85439f1a6b2 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:15:03 +0000 Subject: [PATCH 09/21] =?UTF-8?q?=E6=9B=B4=E6=96=B0RcfDetection=E4=BD=BF?= =?UTF-8?q?=E5=85=B6=E9=80=82=E9=85=8D=E7=89=88=E6=9C=AC?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/EdgeDetectionPicture/RcfDetection.cpp | 286 ++++++++++++++++++ contrib/EdgeDetectionPicture/RcfDetection.h | 61 ++++ 2 files changed, 347 insertions(+) create mode 100644 contrib/EdgeDetectionPicture/RcfDetection.cpp create mode 100644 contrib/EdgeDetectionPicture/RcfDetection.h diff --git a/contrib/EdgeDetectionPicture/RcfDetection.cpp b/contrib/EdgeDetectionPicture/RcfDetection.cpp new file mode 100644 index 000000000..ab1796d72 --- /dev/null +++ b/contrib/EdgeDetectionPicture/RcfDetection.cpp @@ -0,0 +1,286 @@ +/** + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "RcfDetection.h" +#include "MxBase/DeviceManager/DeviceManager.h" +#include "MxBase/Log/Log.h" +#include "../rcfPostProcess/RcfPostProcess.h" +#include +#include +#include "opencv2/opencv.hpp" + +namespace { + const uint32_t YUV_BYTE_NU = 3; + const uint32_t YUV_BYTE_DE = 2; + const uint32_t VPC_H_ALIGN = 2; + const uint32_t channel = 3; + const double ALPHA = 255.0 / 255; +} +void RcfDetection::SetRcfPostProcessConfig(const InitParam &initParam, + std::map &config) +{ + MxBase::ConfigData configData; + const std::string checkTensor = initParam.checkTensor ? "true" : "false"; + configData.SetJsonValue("OUTSIZE_NUM", std::to_string(initParam.outSizeNum)); + configData.SetJsonValue("OUTSIZE", initParam.outSize); + configData.SetJsonValue("RCF_TYPE", std::to_string(initParam.rcfType)); + configData.SetJsonValue("MODEL_TYPE", std::to_string(initParam.modelType)); + configData.SetJsonValue("INPUT_TYPE", std::to_string(initParam.inputType)); + configData.SetJsonValue("CHECK_MODEL", checkTensor); + #ifdef MX_VERSION_5 + auto jsonStr = configData.GetCfgJson().serialize(); + config["postProcessConfigContent"] = jsonStr; + #else + auto jsonStr = configData.GetCfgJson(); + config["postProcessConfigContent"] = jsonStr; + #endif +} + +APP_ERROR RcfDetection::Init(const InitParam &initParam) +{ + deviceId_ = initParam.deviceId; + APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices(); + if (ret != APP_ERR_OK) { + LogError << "Init devices failed, ret=" << ret << "."; + return ret; + } + ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId); + if (ret != APP_ERR_OK) { + LogError << "Set context failed, ret=" << ret << "."; + return ret; + } + dvppWrapper_ = std::make_shared(); + ret = dvppWrapper_->Init(); + if (ret != APP_ERR_OK) { + LogError << "DvppWrapper init failed, ret=" << ret << "."; + return ret; + } + model_ = std::make_shared(); + ret = model_->Init(initParam.modelPath, modelDesc_); + if (ret != APP_ERR_OK) { + LogError << "ModelInferenceProcessor init failed, ret=" << ret << "."; + return ret; + } + + std::map config; + SetRcfPostProcessConfig(initParam, config); + post_ = std::make_shared(); + ret = post_->Init(config); + if (ret != APP_ERR_OK) { + LogError << "RcfPostProcess init failed, ret=" << ret << "."; + return ret; + } + if (ret != APP_ERR_OK) { + LogError << "Failed to load labels, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::DeInit() +{ + dvppWrapper_->DeInit(); + model_->DeInit(); + post_->DeInit(); + MxBase::DeviceManager::GetInstance()->DestroyDevices(); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::ReadImage(const std::string &imgPath, + MxBase::TensorBase &tensor) +{ + MxBase::DvppDataInfo output = {}; + APP_ERROR ret = dvppWrapper_->DvppJpegDecode(imgPath, output); + if (ret != APP_ERR_OK) { + LogError << "DvppWrapper DvppJpegDecode failed, ret=" << ret << "."; + return ret; + } + MxBase::MemoryData memoryData((void*)output.data, output.dataSize, + MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + if (output.heightStride % VPC_H_ALIGN != 0) { + LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; + MxBase::MemoryHelper::MxbsFree(memoryData); + return APP_ERR_COMM_INVALID_PARAM; + } + dvppHeightStride = output.heightStride; + dvppWidthStride = output.widthStride; + std::vector shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; + tensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); + return APP_ERR_OK; +} +APP_ERROR RcfDetection::Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, + uint32_t resizeHeight, uint32_t resizeWidth) +{ + auto shape = inputTensor.GetShape(); + MxBase::DvppDataInfo input = {}; + input.height = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; + input.width = shape[1]; + input.heightStride = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; + input.widthStride = shape[1]; + input.dataSize = inputTensor.GetByteSize(); + input.data = (uint8_t*)inputTensor.GetBuffer(); + MxBase::ResizeConfig resize = {}; + resize.height = resizeHeight; + resize.width = resizeWidth; + MxBase::DvppDataInfo output = {}; + APP_ERROR ret = dvppWrapper_->VpcResize(input, output, resize); + if (ret != APP_ERR_OK) { + LogError << "VpcResize failed, ret=" << ret << "."; + return ret; + } + MxBase::MemoryData memoryData((void*)output.data, output.dataSize, + MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + if (output.heightStride % VPC_H_ALIGN != 0) { + LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; + MxBase::MemoryHelper::MxbsFree(memoryData); + return APP_ERR_COMM_INVALID_PARAM; + } + shape = {1, channel, output.heightStride, output.widthStride}; + outputTensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::Inference(const std::vector &inputs, + std::vector &outputs) +{ + std::vector output={}; + auto dtypes = model_->GetOutputDataType(); + for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) { + std::vector shape = {}; + for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) { + shape.push_back((uint32_t)modelDesc_.outputTensors[i].tensorDims[j]); + } + MxBase::TensorBase tensor(shape, dtypes[i], MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor); + if (ret != APP_ERR_OK) { + LogError << "TensorBaseMalloc failed, ret=" << ret << "."; + return ret; + } + outputs.push_back(tensor); + } + MxBase::DynamicInfo dynamicInfo = {}; + dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH; + auto startTime = std::chrono::high_resolution_clock::now(); + APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo); + auto endTime = std::chrono::high_resolution_clock::now(); + double costMs = std::chrono::duration(endTime - startTime).count(); + LogInfo<< "costMs:"<< costMs; + if (ret != APP_ERR_OK) { + LogError << "ModelInference failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::PostProcess(const MxBase::TensorBase &tensor, + const std::vector &outputs, + std::vector &postProcessOutput) +{ + auto shape = tensor.GetShape(); + MxBase::ResizedImageInfo imgInfo; + imgInfo.widthOriginal = shape[1]; + imgInfo.heightOriginal = shape[0] * YUV_BYTE_DE; + uint32_t widthResize = 512; + uint32_t heightResize = 512; + imgInfo.widthResize = widthResize; + imgInfo.heightResize = heightResize; + imgInfo.resizeType = MxBase::RESIZER_STRETCHING; + std::vector imageInfoVec = {}; + imageInfoVec.push_back(imgInfo); + APP_ERROR ret = post_->Process(outputs, postProcessOutput); + if (ret != APP_ERR_OK) { + LogError << "Process failed, ret=" << ret << "."; + return ret; + } + ret = post_->DeInit(); + if (ret != APP_ERR_OK) { + LogError << "RcfDetection DeInit failed"; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath) +{ + auto shape = inferTensor.GetShape(); + int dim2 = 2; + int dim3 = 3; + uint32_t height = shape[dim2]; + uint32_t width = shape[dim3]; + cv::Mat imgBgr = cv::imread(imgPath); + std::string fileName = imgPath.substr(imgPath.find_last_of("/") + 1); + uint32_t imageWidth = imgBgr.cols; + uint32_t imageHeight = imgBgr.rows; + cv::Mat modelOutput = cv::Mat(height, width, CV_32FC1, inferTensor.GetBuffer()); + cv::Mat grayMat; + cv::Mat resizedMat; + int crop = 5; + cv::Rect myROI(0, 0, imageWidth - crop, imageHeight); + resize(modelOutput, resizedMat, cv::Size(dvppWidthStride, dvppHeightStride), 0, 0, cv::INTER_LINEAR); + resizedMat.convertTo(grayMat, CV_8UC1, ALPHA); + cv::Mat croppedImage = grayMat(myROI); + resize(croppedImage, croppedImage, cv::Size(imageWidth, imageHeight), 0, 0, cv::INTER_LINEAR); + + std::string resultPathName = "result"; + if (access(resultPathName.c_str(), 0) != 0) { + int ret = mkdir(resultPathName.c_str(), S_IRUSR | S_IWUSR | S_IXUSR); + if (ret != 0) { + LogError << "Failed to create result directory: " << resultPathName << ", ret = " << ret; + return APP_ERR_COMM_FAILURE; + } + } + + cv::imwrite("./result/"+fileName, croppedImage); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::Process(const std::string &imgPath) +{ + MxBase::TensorBase inTensor; + APP_ERROR ret = ReadImage(imgPath, inTensor); + if (ret != APP_ERR_OK) { + LogError << "ReadImage failed, ret=" << ret << "."; + return ret; + } + MxBase::TensorBase outTensor; + uint32_t resizeHeight = 512; + uint32_t resizeWidth = 512; + ret = Resize(inTensor, outTensor, resizeHeight, resizeWidth); + if (ret != APP_ERR_OK) { + LogError << "Resize failed, ret=" << ret << "."; + return ret; + } + std::vector inputs = {}; + std::vector outputs = {}; + auto shape = outTensor.GetShape(); + inputs.push_back(outTensor); + ret = Inference(inputs, outputs); + if (ret != APP_ERR_OK) { + LogError << "Inference failed, ret=" << ret << "."; + return ret; + } + std::vector postProcessOutput={}; + ret = PostProcess(inTensor, outputs, postProcessOutput); + if (ret != APP_ERR_OK) { + LogError << "PostProcess failed, ret=" << ret << "."; + return ret; + } + ret = WriteResult(postProcessOutput[0], imgPath); + if (ret != APP_ERR_OK) { + LogError << "Save result failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} diff --git a/contrib/EdgeDetectionPicture/RcfDetection.h b/contrib/EdgeDetectionPicture/RcfDetection.h new file mode 100644 index 000000000..7c0ae6da4 --- /dev/null +++ b/contrib/EdgeDetectionPicture/RcfDetection.h @@ -0,0 +1,61 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef MXBASE_RCFDETECTION_H +#define MXBASE_RCFDETECTION_H + +#include +#include +#include "MxBase/DvppWrapper/DvppWrapper.h" +#include "MxBase/ModelInfer/ModelInferenceProcessor.h" +#include "MxBase/Tensor/TensorContext/TensorContext.h" + +struct InitParam { + uint32_t deviceId = 0; + bool checkTensor = true; + std::string modelPath = ""; + uint32_t outSizeNum = 0; + std::string outSize = ""; + uint32_t rcfType = 0; + uint32_t modelType = 0; + uint32_t inputType = 0; +}; + +class RcfDetection { +public: + APP_ERROR Init(const InitParam &initParam); + APP_ERROR DeInit(); + APP_ERROR Inference(const std::vector &inputs, std::vector &outputs); + APP_ERROR PostProcess(const MxBase::TensorBase &tensor, const std::vector &outputs, + std::vector &postProcessOutput); + APP_ERROR Process(const std::string &imgPath); +protected: + APP_ERROR ReadImage(const std::string &imgPath, MxBase::TensorBase &tensor); + APP_ERROR Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, + uint32_t resizeHeight, uint32_t resizeWidth); + APP_ERROR WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath); + void SetRcfPostProcessConfig(const InitParam &initParam, std::map &config); + +private: + std::shared_ptr dvppWrapper_; + std::shared_ptr model_; + std::shared_ptr post_; + MxBase::ModelDesc modelDesc_ = {}; + uint32_t deviceId_ = 0; + int dvppHeightStride = 0; + int dvppWidthStride = 0; +}; +#endif -- Gitee From d3e96ed7244023009439f4c714e0265842a3200f Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:15:29 +0000 Subject: [PATCH 10/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/rcfDetection/.keep?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/rcfDetection/.keep | 0 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/rcfDetection/.keep diff --git a/contrib/EdgeDetectionPicture/rcfDetection/.keep b/contrib/EdgeDetectionPicture/rcfDetection/.keep deleted file mode 100644 index e69de29bb..000000000 -- Gitee From 2c3f8c16725c766066bccc8207571a1c5971ad70 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:16:18 +0000 Subject: [PATCH 11/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/README.md | 150 ------------------------- 1 file changed, 150 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/README.md diff --git a/contrib/EdgeDetectionPicture/README.md b/contrib/EdgeDetectionPicture/README.md deleted file mode 100644 index e00f69a20..000000000 --- a/contrib/EdgeDetectionPicture/README.md +++ /dev/null @@ -1,150 +0,0 @@ - -# RCF模型边缘检测 - -## 1 介绍 -本开发样例是基于mxBase开发的端到端推理的C++应用程序，可在昇腾芯片上进行 图像边缘提取，并把可视化结果保存到本地。 -其中包含Rcf模型的后处理模块开发。 主要处理流程为： Init > ReadImage >Resize > Inference >PostProcess >DeInit - -#### 1.1 支持的产品 -昇腾310(推理) - -#### 1.2 支持的版本 -本样例配套的CANN版本为7.0.0，MindX SDK版本为5.0.0 -MindX SDK安装前准备可参考《用户指南》，[安装教程](https://gitee.com/ascend/mindxsdk-referenceapps/blob/master/docs/quickStart/1-1%E5%AE%89%E8%A3%85SDK%E5%BC%80%E5%8F%91%E5%A5%97%E4%BB%B6.md) - -#### 1.3 代码目录结构与说明 -本sample工程名称为EdgeDetectionPicture，工程目录如下图所示： - -``` -. -├── model -│ ├── aipp.cfg // 模型转换aipp配置文件 -├── rcfDetection -│ ├── RcfDetection.cpp -│ └── RcfDetection.h -├── rcfPostProcess -│ ├── rcfPostProcess.cpp -│ └── rcfPostProcess.h -├── build.sh -├── main.cpp -├── README.md -├── CMakeLists.txt -└── License -``` - -## 2 环境依赖 -环境依赖软件和版本如下表： - - - -| 软件 | 版本 | 说明 | 获取方式 | -| ------------------- | ------------ | ----------------------------- | ------------------------------------------------------------ | -| mxVision | 5.0.0 | mxVision软件包 | [链接](https://www.hiascend.com/software/Mindx-sdk) | -| Ascend-CANN-toolkit | 7.0.0 | Ascend-cann-toolkit开发套件包 | [链接](https://www.hiascend.com/software/cann/commercial) | -| 操作系统 | Ubuntu 18.04 | 操作系统 | Ubuntu官网获取 | - -在编译运行项目前，需要设置环境变量： - -- 环境变量介绍 - - ``` - . {cann_install_path}/ascend-toolkit/set_env.sh - . {sdk_install_path}/mxVision/set_env.sh - - ``` - - - -## 3 模型转换 - -**步骤1** 模型获取 -下载RCF模型 。[下载地址](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/EdgeDetectionPicture/model.zip) - -**步骤2** 模型存放 -将获取到的RCF模型rcf.prototxt文件和rcf_bsds.caffemodel文件放在edge_detection_picture/model下 - -**步骤3** 执行模型转换命令 - -``` -atc --model=rcf.prototxt --weight=./rcf_bsds.caffemodel --framework=0 --output=rcf --soc_version=Ascend310 --insert_op_conf=./aipp.cfg --input_format=NCHW --output_type=FP32 -``` - -## 4 编译与运行 - -**步骤1** 执行如下编译命令： -bash build.sh - -**步骤2** 进行图像边缘检测 -请自行准备jpg格式的测试图像保存在文件夹中(例如 data/**.jpg)进行边缘检测 -``` -./edge_detection_picture ./data -``` -生成边缘检测图像 result/**.jpg - -## 5 精度测试 -下载开源数据集 BSDS500 [下载地址](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/EdgeDetectionPicture/data.zip), 使用 BSR/BSDS500/data/images/test数据进行测试 - - -(1) 下载开源代码 - -``` shell -git clone https://github.com/Walstruzz/edge_eval_python.git -cd edge_eval_python -``` -(2) 编译cxx - -``` shell -cd cxx/src -source build.sh -``` -(3) 将test中的图像经过边缘检测后的结果保存在result文件夹中 - -``` shell -./edge_detection_picture path/to/BSR/BSDS500/data/images/test/ - -``` - - -(4) 修改检测代码 - -vim mian.py -注释第17行代码 nms_process(model_name_list, result_dir, save_dir, key, file_format), -修改18行为 eval_edge(alg, model_name_list, result_dir, gt_dir, workers) - -vim /impl/edges_eval_dir.py -修改155行为 im = os.path.join(res_dir, "{}.jpg".format(i)) - -vim eval_edge.py -修改14行为 res_dir = result_dir - -(5) 测试精度 - -``` shell -python main.py --result_dir path/to/result --gt_dir paht/to/BSR/BSDS500/data/groundTruth/test - -``` -注: - result_dir: results directory - - gt_dir : ground truth directory - - -## 6常见问题 -### 6.1 精度测试脚本运行时, ModuleNotFoundError报错问题： -问题描述: -运行精度测试脚本过程中, 出现如下类似报错: -``` -********* -File "***/nms_process.py", line 6 in - from impl.toolbox import conv_tri, grad2 -ModuleNotFoundError: No module named 'impl.toolbox' -``` - -解决措施: -``` -方法一: -将环境变量PYTHONPATH里面的/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe去掉, 该路径根据cann实际安装位置不同可能会不一样,须自行确认 - -方法二: -执行命令: unset PYTHONPATH -``` \ No newline at end of file -- Gitee From 60e52abad429d3ce0ef7433007b1855297e474cd Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:16:42 +0000 Subject: [PATCH 12/21] =?UTF-8?q?=E6=9B=B4=E6=96=B0README?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/EdgeDetectionPicture/README.md | 156 +++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100644 contrib/EdgeDetectionPicture/README.md diff --git a/contrib/EdgeDetectionPicture/README.md b/contrib/EdgeDetectionPicture/README.md new file mode 100644 index 000000000..53642883f --- /dev/null +++ b/contrib/EdgeDetectionPicture/README.md @@ -0,0 +1,156 @@ + +# RCF模型边缘检测 + +## 1 介绍 + +#### 1.1 简介 +本开发样例是基于mxBase开发的端到端推理的C++应用程序，可在昇腾芯片上进行 图像边缘提取，并把可视化结果保存到本地。 +其中包含Rcf模型的后处理模块开发。 主要处理流程为： Init > ReadImage >Resize > Inference >PostProcess >DeInit + +#### 1.2 支持的产品 +昇腾310(推理) + +#### 1.3 支持的版本 +本样例配套的CANN版本为7.0.0，MindX SDK版本为5.0.0 +MindX SDK安装前准备可参考《用户指南》，[安装教程](https://gitee.com/ascend/mindxsdk-referenceapps/blob/master/docs/quickStart/1-1%E5%AE%89%E8%A3%85SDK%E5%BC%80%E5%8F%91%E5%A5%97%E4%BB%B6.md) +| 软件 | 版本 | 说明 | +| ------------------- | ------------ | ---------------------------- | +| mxVision | 5.0.0 | mxVision软件包 | +| Ascend-CANN-toolkit | 7.0.0 | Ascend-cann-toolkit开发套件包 | + + +#### 1.4 代码目录结构与说明 +本sample工程名称为EdgeDetectionPicture，工程目录如下图所示： + +``` +. +├── model +│ ├── aipp.cfg // 模型转换aipp配置文件 +├── rcfDetection +│ ├── RcfDetection.cpp +│ └── RcfDetection.h +├── rcfPostProcess +│ ├── rcfPostProcess.cpp +│ └── rcfPostProcess.h +├── build.sh +├── main.cpp +├── README.md +├── CMakeLists.txt +└── License +``` + +## 2 设置环境变量 + +在编译运行项目前，需要设置环境变量： + +- 环境变量介绍 + + ``` + . {cann_install_path}/ascend-toolkit/set_env.sh + . {sdk_install_path}/mxVision/set_env.sh + + ``` + + + +## 3 准备模型 + +**步骤1** 模型获取 +下载RCF模型 。[下载地址](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/EdgeDetectionPicture/model.zip) + +**步骤2** 模型存放 +将获取到的RCF模型rcf.prototxt文件和rcf_bsds.caffemodel文件放在edge_detection_picture/model下 + +**步骤3** 执行模型转换命令 + +``` +atc --model=rcf.prototxt --weight=./rcf_bsds.caffemodel --framework=0 --output=rcf --soc_version=Ascend310 --insert_op_conf=./aipp.cfg --input_format=NCHW --output_type=FP32 +``` + +## 4 编译与运行 + +**步骤1** 执行如下编译命令： +bash build.sh + +**步骤2** 进行图像边缘检测 +请自行准备jpg格式的测试图像保存在文件夹中(例如 data/**.jpg)进行边缘检测 +``` +./edge_detection_picture ./data +``` +生成边缘检测图像 result/**.jpg + +## 5 精度测试 +下载开源数据集 BSDS500 [下载地址](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/EdgeDetectionPicture/data.zip), 使用 BSR/BSDS500/data/images/test数据进行测试 + + +(1) 下载开源代码 + +``` shell +git clone https://github.com/Walstruzz/edge_eval_python.git +cd edge_eval_python +``` +(2) 编译cxx + +``` shell +cd cxx/src +source build.sh +``` +(3) 将test中的图像经过边缘检测后的结果保存在result文件夹中 + +``` shell +./edge_detection_picture path/to/BSR/BSDS500/data/images/test/ + +``` + + +(4) 修改检测代码 + +vim mian.py +注释第17行代码 nms_process(model_name_list, result_dir, save_dir, key, file_format), +修改18行为 eval_edge(alg, model_name_list, result_dir, gt_dir, workers) + +vim /impl/edges_eval_dir.py +修改155行为 im = os.path.join(res_dir, "{}.jpg".format(i)) + +vim eval_edge.py +修改14行为 res_dir = result_dir + +(5) 测试精度 + +``` shell +python main.py --result_dir path/to/result --gt_dir path/to/BSR/BSDS500/data/groundTruth/test + +``` +注: + result_dir: results directory + + gt_dir : ground truth directory + + +## 6常见问题 +### 6.1 精度测试脚本运行时, ModuleNotFoundError报错问题： +问题描述: +运行精度测试脚本过程中, 出现如下类似报错: +``` +********* +File "***/nms_process.py", line 6 in + from impl.toolbox import conv_tri, grad2 +ModuleNotFoundError: No module named 'impl.toolbox' +``` + +解决措施: +``` +方法一: +将环境变量PYTHONPATH里面的/usr/local/Ascend/ascend-toolkit/latest/opp/built-in/op_impl/ai_core/tbe去掉, 该路径根据cann实际安装位置不同可能会不一样,须自行确认 + +方法二: +执行命令: unset PYTHONPATH +``` +### 6.2 检测代码无法修改问题： +问题描述: +修改检测代码中, 出现无法修改问题 + +解决措施: +``` +使用 sudo vim filename进行修改 +``` \ No newline at end of file -- Gitee From 242855048bcea8e0784a24395ed259f284cbd873 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:17:02 +0000 Subject: [PATCH 13/21] =?UTF-8?q?=E6=96=B0=E5=BB=BA=20rcfDetection?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/rcfDetection/.keep | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 contrib/EdgeDetectionPicture/rcfDetection/.keep diff --git a/contrib/EdgeDetectionPicture/rcfDetection/.keep b/contrib/EdgeDetectionPicture/rcfDetection/.keep new file mode 100644 index 000000000..e69de29bb -- Gitee From c38184b05b4a598af37167d258224b5a76718c8e Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:17:31 +0000 Subject: [PATCH 14/21] =?UTF-8?q?=E6=9B=B4=E6=96=B0RcfDetection=E4=BD=BF?= =?UTF-8?q?=E5=85=B6=E9=80=82=E9=85=8D=E7=89=88=E6=9C=AC?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- .../rcfDetection/RcfDetection.cpp | 286 ++++++++++++++++++ .../rcfDetection/RcfDetection.h | 61 ++++ 2 files changed, 347 insertions(+) create mode 100644 contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp create mode 100644 contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h diff --git a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp new file mode 100644 index 000000000..ab1796d72 --- /dev/null +++ b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.cpp @@ -0,0 +1,286 @@ +/** + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "RcfDetection.h" +#include "MxBase/DeviceManager/DeviceManager.h" +#include "MxBase/Log/Log.h" +#include "../rcfPostProcess/RcfPostProcess.h" +#include +#include +#include "opencv2/opencv.hpp" + +namespace { + const uint32_t YUV_BYTE_NU = 3; + const uint32_t YUV_BYTE_DE = 2; + const uint32_t VPC_H_ALIGN = 2; + const uint32_t channel = 3; + const double ALPHA = 255.0 / 255; +} +void RcfDetection::SetRcfPostProcessConfig(const InitParam &initParam, + std::map &config) +{ + MxBase::ConfigData configData; + const std::string checkTensor = initParam.checkTensor ? "true" : "false"; + configData.SetJsonValue("OUTSIZE_NUM", std::to_string(initParam.outSizeNum)); + configData.SetJsonValue("OUTSIZE", initParam.outSize); + configData.SetJsonValue("RCF_TYPE", std::to_string(initParam.rcfType)); + configData.SetJsonValue("MODEL_TYPE", std::to_string(initParam.modelType)); + configData.SetJsonValue("INPUT_TYPE", std::to_string(initParam.inputType)); + configData.SetJsonValue("CHECK_MODEL", checkTensor); + #ifdef MX_VERSION_5 + auto jsonStr = configData.GetCfgJson().serialize(); + config["postProcessConfigContent"] = jsonStr; + #else + auto jsonStr = configData.GetCfgJson(); + config["postProcessConfigContent"] = jsonStr; + #endif +} + +APP_ERROR RcfDetection::Init(const InitParam &initParam) +{ + deviceId_ = initParam.deviceId; + APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices(); + if (ret != APP_ERR_OK) { + LogError << "Init devices failed, ret=" << ret << "."; + return ret; + } + ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId); + if (ret != APP_ERR_OK) { + LogError << "Set context failed, ret=" << ret << "."; + return ret; + } + dvppWrapper_ = std::make_shared(); + ret = dvppWrapper_->Init(); + if (ret != APP_ERR_OK) { + LogError << "DvppWrapper init failed, ret=" << ret << "."; + return ret; + } + model_ = std::make_shared(); + ret = model_->Init(initParam.modelPath, modelDesc_); + if (ret != APP_ERR_OK) { + LogError << "ModelInferenceProcessor init failed, ret=" << ret << "."; + return ret; + } + + std::map config; + SetRcfPostProcessConfig(initParam, config); + post_ = std::make_shared(); + ret = post_->Init(config); + if (ret != APP_ERR_OK) { + LogError << "RcfPostProcess init failed, ret=" << ret << "."; + return ret; + } + if (ret != APP_ERR_OK) { + LogError << "Failed to load labels, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::DeInit() +{ + dvppWrapper_->DeInit(); + model_->DeInit(); + post_->DeInit(); + MxBase::DeviceManager::GetInstance()->DestroyDevices(); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::ReadImage(const std::string &imgPath, + MxBase::TensorBase &tensor) +{ + MxBase::DvppDataInfo output = {}; + APP_ERROR ret = dvppWrapper_->DvppJpegDecode(imgPath, output); + if (ret != APP_ERR_OK) { + LogError << "DvppWrapper DvppJpegDecode failed, ret=" << ret << "."; + return ret; + } + MxBase::MemoryData memoryData((void*)output.data, output.dataSize, + MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + if (output.heightStride % VPC_H_ALIGN != 0) { + LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; + MxBase::MemoryHelper::MxbsFree(memoryData); + return APP_ERR_COMM_INVALID_PARAM; + } + dvppHeightStride = output.heightStride; + dvppWidthStride = output.widthStride; + std::vector shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; + tensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); + return APP_ERR_OK; +} +APP_ERROR RcfDetection::Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, + uint32_t resizeHeight, uint32_t resizeWidth) +{ + auto shape = inputTensor.GetShape(); + MxBase::DvppDataInfo input = {}; + input.height = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; + input.width = shape[1]; + input.heightStride = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; + input.widthStride = shape[1]; + input.dataSize = inputTensor.GetByteSize(); + input.data = (uint8_t*)inputTensor.GetBuffer(); + MxBase::ResizeConfig resize = {}; + resize.height = resizeHeight; + resize.width = resizeWidth; + MxBase::DvppDataInfo output = {}; + APP_ERROR ret = dvppWrapper_->VpcResize(input, output, resize); + if (ret != APP_ERR_OK) { + LogError << "VpcResize failed, ret=" << ret << "."; + return ret; + } + MxBase::MemoryData memoryData((void*)output.data, output.dataSize, + MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + if (output.heightStride % VPC_H_ALIGN != 0) { + LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; + MxBase::MemoryHelper::MxbsFree(memoryData); + return APP_ERR_COMM_INVALID_PARAM; + } + shape = {1, channel, output.heightStride, output.widthStride}; + outputTensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::Inference(const std::vector &inputs, + std::vector &outputs) +{ + std::vector output={}; + auto dtypes = model_->GetOutputDataType(); + for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) { + std::vector shape = {}; + for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) { + shape.push_back((uint32_t)modelDesc_.outputTensors[i].tensorDims[j]); + } + MxBase::TensorBase tensor(shape, dtypes[i], MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); + APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor); + if (ret != APP_ERR_OK) { + LogError << "TensorBaseMalloc failed, ret=" << ret << "."; + return ret; + } + outputs.push_back(tensor); + } + MxBase::DynamicInfo dynamicInfo = {}; + dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH; + auto startTime = std::chrono::high_resolution_clock::now(); + APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo); + auto endTime = std::chrono::high_resolution_clock::now(); + double costMs = std::chrono::duration(endTime - startTime).count(); + LogInfo<< "costMs:"<< costMs; + if (ret != APP_ERR_OK) { + LogError << "ModelInference failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::PostProcess(const MxBase::TensorBase &tensor, + const std::vector &outputs, + std::vector &postProcessOutput) +{ + auto shape = tensor.GetShape(); + MxBase::ResizedImageInfo imgInfo; + imgInfo.widthOriginal = shape[1]; + imgInfo.heightOriginal = shape[0] * YUV_BYTE_DE; + uint32_t widthResize = 512; + uint32_t heightResize = 512; + imgInfo.widthResize = widthResize; + imgInfo.heightResize = heightResize; + imgInfo.resizeType = MxBase::RESIZER_STRETCHING; + std::vector imageInfoVec = {}; + imageInfoVec.push_back(imgInfo); + APP_ERROR ret = post_->Process(outputs, postProcessOutput); + if (ret != APP_ERR_OK) { + LogError << "Process failed, ret=" << ret << "."; + return ret; + } + ret = post_->DeInit(); + if (ret != APP_ERR_OK) { + LogError << "RcfDetection DeInit failed"; + return ret; + } + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath) +{ + auto shape = inferTensor.GetShape(); + int dim2 = 2; + int dim3 = 3; + uint32_t height = shape[dim2]; + uint32_t width = shape[dim3]; + cv::Mat imgBgr = cv::imread(imgPath); + std::string fileName = imgPath.substr(imgPath.find_last_of("/") + 1); + uint32_t imageWidth = imgBgr.cols; + uint32_t imageHeight = imgBgr.rows; + cv::Mat modelOutput = cv::Mat(height, width, CV_32FC1, inferTensor.GetBuffer()); + cv::Mat grayMat; + cv::Mat resizedMat; + int crop = 5; + cv::Rect myROI(0, 0, imageWidth - crop, imageHeight); + resize(modelOutput, resizedMat, cv::Size(dvppWidthStride, dvppHeightStride), 0, 0, cv::INTER_LINEAR); + resizedMat.convertTo(grayMat, CV_8UC1, ALPHA); + cv::Mat croppedImage = grayMat(myROI); + resize(croppedImage, croppedImage, cv::Size(imageWidth, imageHeight), 0, 0, cv::INTER_LINEAR); + + std::string resultPathName = "result"; + if (access(resultPathName.c_str(), 0) != 0) { + int ret = mkdir(resultPathName.c_str(), S_IRUSR | S_IWUSR | S_IXUSR); + if (ret != 0) { + LogError << "Failed to create result directory: " << resultPathName << ", ret = " << ret; + return APP_ERR_COMM_FAILURE; + } + } + + cv::imwrite("./result/"+fileName, croppedImage); + return APP_ERR_OK; +} + +APP_ERROR RcfDetection::Process(const std::string &imgPath) +{ + MxBase::TensorBase inTensor; + APP_ERROR ret = ReadImage(imgPath, inTensor); + if (ret != APP_ERR_OK) { + LogError << "ReadImage failed, ret=" << ret << "."; + return ret; + } + MxBase::TensorBase outTensor; + uint32_t resizeHeight = 512; + uint32_t resizeWidth = 512; + ret = Resize(inTensor, outTensor, resizeHeight, resizeWidth); + if (ret != APP_ERR_OK) { + LogError << "Resize failed, ret=" << ret << "."; + return ret; + } + std::vector inputs = {}; + std::vector outputs = {}; + auto shape = outTensor.GetShape(); + inputs.push_back(outTensor); + ret = Inference(inputs, outputs); + if (ret != APP_ERR_OK) { + LogError << "Inference failed, ret=" << ret << "."; + return ret; + } + std::vector postProcessOutput={}; + ret = PostProcess(inTensor, outputs, postProcessOutput); + if (ret != APP_ERR_OK) { + LogError << "PostProcess failed, ret=" << ret << "."; + return ret; + } + ret = WriteResult(postProcessOutput[0], imgPath); + if (ret != APP_ERR_OK) { + LogError << "Save result failed, ret=" << ret << "."; + return ret; + } + return APP_ERR_OK; +} diff --git a/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h new file mode 100644 index 000000000..7c0ae6da4 --- /dev/null +++ b/contrib/EdgeDetectionPicture/rcfDetection/RcfDetection.h @@ -0,0 +1,61 @@ +/* + * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef MXBASE_RCFDETECTION_H +#define MXBASE_RCFDETECTION_H + +#include +#include +#include "MxBase/DvppWrapper/DvppWrapper.h" +#include "MxBase/ModelInfer/ModelInferenceProcessor.h" +#include "MxBase/Tensor/TensorContext/TensorContext.h" + +struct InitParam { + uint32_t deviceId = 0; + bool checkTensor = true; + std::string modelPath = ""; + uint32_t outSizeNum = 0; + std::string outSize = ""; + uint32_t rcfType = 0; + uint32_t modelType = 0; + uint32_t inputType = 0; +}; + +class RcfDetection { +public: + APP_ERROR Init(const InitParam &initParam); + APP_ERROR DeInit(); + APP_ERROR Inference(const std::vector &inputs, std::vector &outputs); + APP_ERROR PostProcess(const MxBase::TensorBase &tensor, const std::vector &outputs, + std::vector &postProcessOutput); + APP_ERROR Process(const std::string &imgPath); +protected: + APP_ERROR ReadImage(const std::string &imgPath, MxBase::TensorBase &tensor); + APP_ERROR Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, + uint32_t resizeHeight, uint32_t resizeWidth); + APP_ERROR WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath); + void SetRcfPostProcessConfig(const InitParam &initParam, std::map &config); + +private: + std::shared_ptr dvppWrapper_; + std::shared_ptr model_; + std::shared_ptr post_; + MxBase::ModelDesc modelDesc_ = {}; + uint32_t deviceId_ = 0; + int dvppHeightStride = 0; + int dvppWidthStride = 0; +}; +#endif -- Gitee From 6bbe86b9a7c771bb196f2ff04f28c24f1a99123d Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:17:36 +0000 Subject: [PATCH 15/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/rcfDetection/.keep?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/rcfDetection/.keep | 0 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/rcfDetection/.keep diff --git a/contrib/EdgeDetectionPicture/rcfDetection/.keep b/contrib/EdgeDetectionPicture/rcfDetection/.keep deleted file mode 100644 index e69de29bb..000000000 -- Gitee From 6d94ab464e821fead4732ba4d0e45c6f0bd9bb50 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:17:45 +0000 Subject: [PATCH 16/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/RcfDetection.h?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/RcfDetection.h | 61 --------------------- 1 file changed, 61 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/RcfDetection.h diff --git a/contrib/EdgeDetectionPicture/RcfDetection.h b/contrib/EdgeDetectionPicture/RcfDetection.h deleted file mode 100644 index 7c0ae6da4..000000000 --- a/contrib/EdgeDetectionPicture/RcfDetection.h +++ /dev/null @@ -1,61 +0,0 @@ -/* - * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#ifndef MXBASE_RCFDETECTION_H -#define MXBASE_RCFDETECTION_H - -#include -#include -#include "MxBase/DvppWrapper/DvppWrapper.h" -#include "MxBase/ModelInfer/ModelInferenceProcessor.h" -#include "MxBase/Tensor/TensorContext/TensorContext.h" - -struct InitParam { - uint32_t deviceId = 0; - bool checkTensor = true; - std::string modelPath = ""; - uint32_t outSizeNum = 0; - std::string outSize = ""; - uint32_t rcfType = 0; - uint32_t modelType = 0; - uint32_t inputType = 0; -}; - -class RcfDetection { -public: - APP_ERROR Init(const InitParam &initParam); - APP_ERROR DeInit(); - APP_ERROR Inference(const std::vector &inputs, std::vector &outputs); - APP_ERROR PostProcess(const MxBase::TensorBase &tensor, const std::vector &outputs, - std::vector &postProcessOutput); - APP_ERROR Process(const std::string &imgPath); -protected: - APP_ERROR ReadImage(const std::string &imgPath, MxBase::TensorBase &tensor); - APP_ERROR Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, - uint32_t resizeHeight, uint32_t resizeWidth); - APP_ERROR WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath); - void SetRcfPostProcessConfig(const InitParam &initParam, std::map &config); - -private: - std::shared_ptr dvppWrapper_; - std::shared_ptr model_; - std::shared_ptr post_; - MxBase::ModelDesc modelDesc_ = {}; - uint32_t deviceId_ = 0; - int dvppHeightStride = 0; - int dvppWidthStride = 0; -}; -#endif -- Gitee From 092009ea6922a7d34c3d56db2ca0cdb7b8110205 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:17:51 +0000 Subject: [PATCH 17/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/EdgeDetectionPicture/RcfDetection.cpp?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/EdgeDetectionPicture/RcfDetection.cpp | 286 ------------------ 1 file changed, 286 deletions(-) delete mode 100644 contrib/EdgeDetectionPicture/RcfDetection.cpp diff --git a/contrib/EdgeDetectionPicture/RcfDetection.cpp b/contrib/EdgeDetectionPicture/RcfDetection.cpp deleted file mode 100644 index ab1796d72..000000000 --- a/contrib/EdgeDetectionPicture/RcfDetection.cpp +++ /dev/null @@ -1,286 +0,0 @@ -/** - * Copyright(C) 2021. Huawei Technologies Co.,Ltd. All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#include "RcfDetection.h" -#include "MxBase/DeviceManager/DeviceManager.h" -#include "MxBase/Log/Log.h" -#include "../rcfPostProcess/RcfPostProcess.h" -#include -#include -#include "opencv2/opencv.hpp" - -namespace { - const uint32_t YUV_BYTE_NU = 3; - const uint32_t YUV_BYTE_DE = 2; - const uint32_t VPC_H_ALIGN = 2; - const uint32_t channel = 3; - const double ALPHA = 255.0 / 255; -} -void RcfDetection::SetRcfPostProcessConfig(const InitParam &initParam, - std::map &config) -{ - MxBase::ConfigData configData; - const std::string checkTensor = initParam.checkTensor ? "true" : "false"; - configData.SetJsonValue("OUTSIZE_NUM", std::to_string(initParam.outSizeNum)); - configData.SetJsonValue("OUTSIZE", initParam.outSize); - configData.SetJsonValue("RCF_TYPE", std::to_string(initParam.rcfType)); - configData.SetJsonValue("MODEL_TYPE", std::to_string(initParam.modelType)); - configData.SetJsonValue("INPUT_TYPE", std::to_string(initParam.inputType)); - configData.SetJsonValue("CHECK_MODEL", checkTensor); - #ifdef MX_VERSION_5 - auto jsonStr = configData.GetCfgJson().serialize(); - config["postProcessConfigContent"] = jsonStr; - #else - auto jsonStr = configData.GetCfgJson(); - config["postProcessConfigContent"] = jsonStr; - #endif -} - -APP_ERROR RcfDetection::Init(const InitParam &initParam) -{ - deviceId_ = initParam.deviceId; - APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices(); - if (ret != APP_ERR_OK) { - LogError << "Init devices failed, ret=" << ret << "."; - return ret; - } - ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId); - if (ret != APP_ERR_OK) { - LogError << "Set context failed, ret=" << ret << "."; - return ret; - } - dvppWrapper_ = std::make_shared(); - ret = dvppWrapper_->Init(); - if (ret != APP_ERR_OK) { - LogError << "DvppWrapper init failed, ret=" << ret << "."; - return ret; - } - model_ = std::make_shared(); - ret = model_->Init(initParam.modelPath, modelDesc_); - if (ret != APP_ERR_OK) { - LogError << "ModelInferenceProcessor init failed, ret=" << ret << "."; - return ret; - } - - std::map config; - SetRcfPostProcessConfig(initParam, config); - post_ = std::make_shared(); - ret = post_->Init(config); - if (ret != APP_ERR_OK) { - LogError << "RcfPostProcess init failed, ret=" << ret << "."; - return ret; - } - if (ret != APP_ERR_OK) { - LogError << "Failed to load labels, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::DeInit() -{ - dvppWrapper_->DeInit(); - model_->DeInit(); - post_->DeInit(); - MxBase::DeviceManager::GetInstance()->DestroyDevices(); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::ReadImage(const std::string &imgPath, - MxBase::TensorBase &tensor) -{ - MxBase::DvppDataInfo output = {}; - APP_ERROR ret = dvppWrapper_->DvppJpegDecode(imgPath, output); - if (ret != APP_ERR_OK) { - LogError << "DvppWrapper DvppJpegDecode failed, ret=" << ret << "."; - return ret; - } - MxBase::MemoryData memoryData((void*)output.data, output.dataSize, - MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - if (output.heightStride % VPC_H_ALIGN != 0) { - LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; - MxBase::MemoryHelper::MxbsFree(memoryData); - return APP_ERR_COMM_INVALID_PARAM; - } - dvppHeightStride = output.heightStride; - dvppWidthStride = output.widthStride; - std::vector shape = {output.heightStride * YUV_BYTE_NU / YUV_BYTE_DE, output.widthStride}; - tensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); - return APP_ERR_OK; -} -APP_ERROR RcfDetection::Resize(const MxBase::TensorBase &inputTensor, MxBase::TensorBase &outputTensor, - uint32_t resizeHeight, uint32_t resizeWidth) -{ - auto shape = inputTensor.GetShape(); - MxBase::DvppDataInfo input = {}; - input.height = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; - input.width = shape[1]; - input.heightStride = (uint32_t)shape[0] * YUV_BYTE_DE / YUV_BYTE_NU; - input.widthStride = shape[1]; - input.dataSize = inputTensor.GetByteSize(); - input.data = (uint8_t*)inputTensor.GetBuffer(); - MxBase::ResizeConfig resize = {}; - resize.height = resizeHeight; - resize.width = resizeWidth; - MxBase::DvppDataInfo output = {}; - APP_ERROR ret = dvppWrapper_->VpcResize(input, output, resize); - if (ret != APP_ERR_OK) { - LogError << "VpcResize failed, ret=" << ret << "."; - return ret; - } - MxBase::MemoryData memoryData((void*)output.data, output.dataSize, - MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - if (output.heightStride % VPC_H_ALIGN != 0) { - LogError << "Output data height(" << output.heightStride << ") can't be divided by " << VPC_H_ALIGN << "."; - MxBase::MemoryHelper::MxbsFree(memoryData); - return APP_ERR_COMM_INVALID_PARAM; - } - shape = {1, channel, output.heightStride, output.widthStride}; - outputTensor = MxBase::TensorBase(memoryData, false, shape, MxBase::TENSOR_DTYPE_UINT8); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::Inference(const std::vector &inputs, - std::vector &outputs) -{ - std::vector output={}; - auto dtypes = model_->GetOutputDataType(); - for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) { - std::vector shape = {}; - for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) { - shape.push_back((uint32_t)modelDesc_.outputTensors[i].tensorDims[j]); - } - MxBase::TensorBase tensor(shape, dtypes[i], MxBase::MemoryData::MemoryType::MEMORY_DEVICE, deviceId_); - APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor); - if (ret != APP_ERR_OK) { - LogError << "TensorBaseMalloc failed, ret=" << ret << "."; - return ret; - } - outputs.push_back(tensor); - } - MxBase::DynamicInfo dynamicInfo = {}; - dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH; - auto startTime = std::chrono::high_resolution_clock::now(); - APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo); - auto endTime = std::chrono::high_resolution_clock::now(); - double costMs = std::chrono::duration(endTime - startTime).count(); - LogInfo<< "costMs:"<< costMs; - if (ret != APP_ERR_OK) { - LogError << "ModelInference failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::PostProcess(const MxBase::TensorBase &tensor, - const std::vector &outputs, - std::vector &postProcessOutput) -{ - auto shape = tensor.GetShape(); - MxBase::ResizedImageInfo imgInfo; - imgInfo.widthOriginal = shape[1]; - imgInfo.heightOriginal = shape[0] * YUV_BYTE_DE; - uint32_t widthResize = 512; - uint32_t heightResize = 512; - imgInfo.widthResize = widthResize; - imgInfo.heightResize = heightResize; - imgInfo.resizeType = MxBase::RESIZER_STRETCHING; - std::vector imageInfoVec = {}; - imageInfoVec.push_back(imgInfo); - APP_ERROR ret = post_->Process(outputs, postProcessOutput); - if (ret != APP_ERR_OK) { - LogError << "Process failed, ret=" << ret << "."; - return ret; - } - ret = post_->DeInit(); - if (ret != APP_ERR_OK) { - LogError << "RcfDetection DeInit failed"; - return ret; - } - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::WriteResult(MxBase::TensorBase &inferTensor, const std::string &imgPath) -{ - auto shape = inferTensor.GetShape(); - int dim2 = 2; - int dim3 = 3; - uint32_t height = shape[dim2]; - uint32_t width = shape[dim3]; - cv::Mat imgBgr = cv::imread(imgPath); - std::string fileName = imgPath.substr(imgPath.find_last_of("/") + 1); - uint32_t imageWidth = imgBgr.cols; - uint32_t imageHeight = imgBgr.rows; - cv::Mat modelOutput = cv::Mat(height, width, CV_32FC1, inferTensor.GetBuffer()); - cv::Mat grayMat; - cv::Mat resizedMat; - int crop = 5; - cv::Rect myROI(0, 0, imageWidth - crop, imageHeight); - resize(modelOutput, resizedMat, cv::Size(dvppWidthStride, dvppHeightStride), 0, 0, cv::INTER_LINEAR); - resizedMat.convertTo(grayMat, CV_8UC1, ALPHA); - cv::Mat croppedImage = grayMat(myROI); - resize(croppedImage, croppedImage, cv::Size(imageWidth, imageHeight), 0, 0, cv::INTER_LINEAR); - - std::string resultPathName = "result"; - if (access(resultPathName.c_str(), 0) != 0) { - int ret = mkdir(resultPathName.c_str(), S_IRUSR | S_IWUSR | S_IXUSR); - if (ret != 0) { - LogError << "Failed to create result directory: " << resultPathName << ", ret = " << ret; - return APP_ERR_COMM_FAILURE; - } - } - - cv::imwrite("./result/"+fileName, croppedImage); - return APP_ERR_OK; -} - -APP_ERROR RcfDetection::Process(const std::string &imgPath) -{ - MxBase::TensorBase inTensor; - APP_ERROR ret = ReadImage(imgPath, inTensor); - if (ret != APP_ERR_OK) { - LogError << "ReadImage failed, ret=" << ret << "."; - return ret; - } - MxBase::TensorBase outTensor; - uint32_t resizeHeight = 512; - uint32_t resizeWidth = 512; - ret = Resize(inTensor, outTensor, resizeHeight, resizeWidth); - if (ret != APP_ERR_OK) { - LogError << "Resize failed, ret=" << ret << "."; - return ret; - } - std::vector inputs = {}; - std::vector outputs = {}; - auto shape = outTensor.GetShape(); - inputs.push_back(outTensor); - ret = Inference(inputs, outputs); - if (ret != APP_ERR_OK) { - LogError << "Inference failed, ret=" << ret << "."; - return ret; - } - std::vector postProcessOutput={}; - ret = PostProcess(inTensor, outputs, postProcessOutput); - if (ret != APP_ERR_OK) { - LogError << "PostProcess failed, ret=" << ret << "."; - return ret; - } - ret = WriteResult(postProcessOutput[0], imgPath); - if (ret != APP_ERR_OK) { - LogError << "Save result failed, ret=" << ret << "."; - return ret; - } - return APP_ERR_OK; -} -- Gitee From 7a361e132d66248b60ff1a52f2a0473b351a51aa Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Sat, 12 Oct 2024 07:18:44 +0000 Subject: [PATCH 18/21] update contrib/EdgeDetectionPicture/README.md. Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/EdgeDetectionPicture/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/contrib/EdgeDetectionPicture/README.md b/contrib/EdgeDetectionPicture/README.md index 53642883f..ec2f72b99 100644 --- a/contrib/EdgeDetectionPicture/README.md +++ b/contrib/EdgeDetectionPicture/README.md @@ -19,7 +19,7 @@ MindX SDK安装前准备可参考《用户指南》，[安装教程](https://git | Ascend-CANN-toolkit | 7.0.0 | Ascend-cann-toolkit开发套件包 | -#### 1.4 代码目录结构与说明 +#### 1.4 代码目录结构说明 本sample工程名称为EdgeDetectionPicture，工程目录如下图所示： ``` @@ -48,7 +48,6 @@ MindX SDK安装前准备可参考《用户指南》，[安装教程](https://git ``` . {cann_install_path}/ascend-toolkit/set_env.sh . {sdk_install_path}/mxVision/set_env.sh - ``` -- Gitee From 273fc27b82152c42cf8630a18a4f42db87589f94 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 01:50:47 +0000 Subject: [PATCH 19/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/TSM/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/TSM/README.md | 341 ------------------------------------------ 1 file changed, 341 deletions(-) delete mode 100644 contrib/TSM/README.md diff --git a/contrib/TSM/README.md b/contrib/TSM/README.md deleted file mode 100644 index 7356a5817..000000000 --- a/contrib/TSM/README.md +++ /dev/null @@ -1,341 +0,0 @@ -# TSM视频分类参考设计 - -## 1 介绍 -使用TSM模型，基于Kinetics-400数据集，在MindX SDK环境下实现视频分类功能。将测试视频传入脚本进行前处理，模型推理，后处理等功能，最终得到模型推理的精度和性能。 - -### 1.1 支持的产品 - -以昇腾Atlas310卡为主要的硬件平台 - -### 1.2 支持的版本 - -CANN：7.0.RC1 - -SDK：mxVision 5.0.RC3（可通过cat SDK目录下的 version.info 查看） - -### 1.3 软件方案介绍 - -项目主要由离线精度测试文件，在线功能测试文件，离线单视频推理性能测试文件，模型文件，测试数据集预处理文件组成。 - -### 1.4 代码目录结构与说明 - -```text -├── TSM - ├── README.md // 所有模型相关说明 - ├── model - ├── onnx2om.sh // 转om模型脚本 - ├── onnx2om1.sh // 在线模型转om模型脚本 - ├── label - ├── kinetics_val.csv // label文件 - ├── download_data - ├── k400_extractor.sh // 解压数据集脚本 - ├── offline.png // 离线推理技术实现流程 - ├── online.png // 在线推理技术实现流程 - ├── online_infer.py // 在线推理精度脚本 - ├── offline_infer.py // 离线推理精度脚本 - ├── speed.py // 离线单视频推理NPU性能脚本 - ├── speed_gpu.py // 离线单视频推理GPU性能脚本 -``` - -### 1.5技术实现流程 - -离线推理流程： - -![离线推理流程](./offline.png) - -在线推理流程： - -![离线推理流程](./online.png) - -### 1.6特性及适用场景 - -离线模型： - -本案例中的 TSM 模型适用于Kinetics数据集中的400类视频分类，并可以返回测试集视频的精度值及单视频识别的种类、性能。 - -在以下两种情况视频分类情况不太好：1. 视频长度过短（小于3s）。 2. 视频帧率过低。 - -在线模型： - -本案例中的在线模型适用于26中手势识别，并可以返回识别手势的名称。 - -## 2 环境依赖 - -推荐系统为ubuntu 18.04，环境依赖软件和版本如下表 - -| 软件名称 | 版本 | -|----------|--------| -| cmake | 3.5+ | -| mxVision | 5.0.RC3 | -| Python | 3.9 | -| torch | 1.10.0 | -| ffmpeg | 4.2.1 | - -- 环境变量搭建 - -在运行项目前，需要设置环境变量： - -MindSDK 环境变量： - -```Shell -. ${SDK-path}/set_env.sh -``` - -CANN 环境变量： - -```Shell -. ${ascend-toolkit-path}/set_env.sh -``` - -环境变量介绍 - -SDK-path: mxVision SDK 安装路径 - -ascend-toolkit-path: CANN 安装路径。 - -下载[ffmpeg](https://github.com/FFmpeg/FFmpeg/archive/n4.2.1.tar.gz)，解压进入并执行以下命令安装： - -```Shell -./configure --prefix=/usr/local/ffmpeg --enable-shared -make -j -make install -``` - -安装完毕后导入环境变量 - -```Shell -export PATH=/usr/local/ffmpeg/bin:$PATH -export LD_LIBRARY_PATH=/usr/local/ffmpeg/lib:$LD_LIBRARY_PATH -``` - -## 3 离线推理 - -**步骤1** Kinetics-400数据集下载 - -参考[Kinetics-400 数据准备](https://github.com/PaddlePaddle/PaddleVideo/blob/develop/docs/zh-CN/dataset/k400.md#%E4%B8%8B%E8%BD%BDvideo%E6%95%B0%E6%8D%AE)中的脚本下载操作，在代码根目录的"download_data"目录下准备"download.sh"数据集下载脚本和"val_link.list"验证集链接列表文件。 - -```text -├── TSM - ├── download_data - ├── download.sh // 下载数据集脚本 - ├── k400_extractor.sh // 解压数据集脚本 - ├── val_link.list -``` - -进入代码根目录的"download_data"目录下，执行以下命令下载数据集压缩包val_part1.tar、val_part2.tar、val_part3.tar： - -```Shell -bash download.sh val_link.list -``` - -然后执行以下命令解压数据集到代码根目录下： - -```Shell -bash k400_extractor.sh -``` - -数据集结构如下： - -```text -├── TSM - ├── data - ├── abseiling - ├── air_drumming - ├── ... - ├── zumba -``` - -**步骤2** 数据集预处理 - -1、视频抽帧 - -在代码根目录执行以下命令创建所需目录： - -```Shell -mkdir tools -mkdir ops -``` - -下载[“temporal-shift-module-master.zip”](https://github.com/mit-han-lab/temporal-shift-module/tree/master)代码包并上传服务器解压，将代码包中"tools"目录下的"vid2img_kinetics.py"、"gen_label_kinetics.py"、"kinetics_label_map.txt"三个文件拷贝至参考设计代码根目录的“tools”目录下。 - -```text -├── TSM - ├── tools - ├── gen_label_kinetics.py // label生成脚本 - ├── vid2img_kinetics.py // 视频抽帧脚本 - ├── kinetics_label_map.txt -``` - -将代码包中"ops"目录下的"basic_ops.py"、"dataset.py"、"dataset_config.py"、"models.py"、"temporal_shift.py"、"transforms.py"六个文件拷贝至参考设计代码根目录的“ops”目录下。 - -```text - ├── ops - ├── basic_ops.py - ├── dataset.py // 数据集构建脚本 - ├── dataset_config.py // 数据集配置脚本 - ├── models.py // 模型搭建脚本 - ├── temporal_shift.py - ├── transforms.py -``` - -修改“tools”目录下的 vid2img_kinetics.py 内容，将77、78行注释。 - -```text - -77行 #class_name = 'test' -78行 #class_process(dir_path, dst_dir_path, class_name) - -``` - -在参考设计代码根目录下，执行以下命令对数据集视频进行抽帧并生成图片： - -```shell -mkdir dataset -cd ./tools -python3 vid2img_kinetics.py [video_path] [image_path] -e.g. -python3 vid2img_kinetics.py ../data ../dataset/ -``` - -修改“tools”目录下gen_label_kinetics.py 内容。 - -```text - -# 11行 dataset_path = '../dataset' # 放视频抽帧后的图片路径 -# 12行 label_path = '../label' # 存放label路径 -# 25行 files_input = ['kinetics_val.csv'] -# 26行 files_output = ['val_videofolder.txt'] -# 37行 folders.append(items[1]) -# 57行 output.append('%s %d %d'%(os.path.join('../dataset/',os.path.join(categories_list[i], curFolder)), len(dir_files), curIDX)) - -``` - -在“tools”目录下，执行以下命令生成标签文件： - -```shell -python3 gen_label_kinetics.py -``` - -**步骤3** 模型转换 - -下载[离线模型](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/TSM/offline_models.zip) TSM.onnx, 将下载好的模型放在“${TSM代码根目录}/model”目录下。 - -将模型转换为om模型，在“model”目录下，执行以下命令生成om模型 - -```shell -bash onnx2om.sh -``` - -**步骤4** 精度测试 - -修改${TSM代码根目录}/ops/dataset_config.py 脚本中参数root_data、filename_imglist_train和filename_imglist_val，若仅进行离线精度测试则可忽略filename_imglist_train设置。 - -```shell -import os - -ROOT_DATASET = './labels/' # 标签文件所在路径 - -... - -def return_kinetics(modality): - filename_categories = 400 - if modality == 'RGB': - root_data = ROOT_DATASET # 训练集根目录 - filename_imglist_train = 'train_videofolder.txt' # 训练数据集标签 - filename_imglist_val = 'val_videofolder.txt' # 测试数据集标签 - prefix = 'img_{:05d}.jpg' - else: - raise NotImplementedError('no such modality:' + modality) - return filename_categories, filename_imglist_train, filename_imglist_val, root_data, prefix -``` - -在参考设计代码根目录下，运行精度测试脚本 - -```shell -python3 offline_infer.py kinetics -``` - -原模型精度值为71.1%，实测精度值为71.01%，符合精度偏差范围，精度达标。 - -**步骤5** 性能测试 - -将用来测试的单视频放在参考设计代码根目录下，如视频“test_speed.mp4”，运行性能测试脚本 - -修改speed_gpu.py与speed.py参数，'./test_speed.mp4'为测试视频，测试视频类别需在Kinetics-400数据集的400个种类内且视频长度至少为3s。 - -```python -def main(): - cmd = 'ffmpeg -i \"{}\" -threads 1 -vf scale=-1:331 -q:v 0 \"{}/img_%05d.jpg\"'.format('./test_speed.mp4', './image') - subprocess.call(cmd, shell=True, - stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) - files = os.listdir(r"./image/") -``` - -GPU性能(Tesla_V100S_PCIE_32GB) - -在参考设计代码根目录下，运行GPU性能测试脚本 - -```shell -python3 speed_gpu.py kinetics --test_segments=8 --test_crops=1 --batch_size=1 -``` - -注：speed_gpu.py脚本需在GPU环境上运行，NPU环境无法运行。 - -得到单视频纯推理性能为0.08sec/video - -SDK性能 - -在参考设计代码根目录下，运行SDK性能测试脚本 - -```shell -python3 speed.py -``` - -注：speed.py脚本需在NPU环境上运行。 - -得到单视频纯推理性能为0.189sec/video - -## 4 在线推理 - -**步骤1** 安装[视频流工具](https://gitee.com/ascend/docs-openmind/blob/master/guide/mindx/sdk/tutorials/reference_material/Live555%E7%A6%BB%E7%BA%BF%E8%A7%86%E9%A2%91%E8%BD%ACRTSP%E8%AF%B4%E6%98%8E%E6%96%87%E6%A1%A3.md) - -**步骤2** 生成视频流 - -根据提示当前只支持部分视频格式，并不支持.mp4后缀的文件，但可以通过ffmpeg转换生成[ffmpeg安装教程](https://gitee.com/ascend/docs-openmind/blob/master/guide/mindx/sdk/tutorials/reference_material/pc%E7%AB%AFffmpeg%E5%AE%89%E8%A3%85%E6%95%99%E7%A8%8B.md)，如下所示为MP4转换为h.264命令： - -使用ffmpeg工具将带有手势的“jester.mp4”的mp4格式视频转换生成为“jester.264”的264格式视频： - -```shell -ffmpeg -i jester.mp4 -vcodec h264 -bf 0 -g 25 -r 10 -s 1280*720 -an -f h264 jester.264 - -//-bf B帧数目控制，-g 关键帧间隔控制，-s 分辨率控制 -an关闭音频， -r 指定帧率 -``` - -使用live555生成视频流。 - -**步骤3** 模型转换 - -下载[在线模型](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/TSM/online_models.zip) jester.onnx - -将下载好的模型放在参考设计代码根目录的“model”目录下。 - -将模型转换为om模型，在“model”目录下，运行脚本生成om模型 - -```shell -bash onnx2om1.sh -``` - -**步骤4** 程序测试 - -```shell -python3 online_infer.py -``` - -修改参数，'ip:port/jester.264'为测试视频流，其中ip为起流的机器ip地址，port为起流的机器端口地址，jester.264为测试视频jester.mp4通过ffmpeg转换后的视频。 - -```python -def video2img(): - cmd = 'ffmpeg -i \"{}\" -threads 1 -vf scale=-1:331 -q:v 0 \"{}/img_%05d.jpg\"'.format('rtsp://ip:port/jester.264', './image') - subprocess.call(cmd, shell=True, - stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) -``` -- Gitee From b60f68bfb33db81fb87ceb3ffa6a27c3e8aa1683 Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Mon, 14 Oct 2024 01:51:20 +0000 Subject: [PATCH 20/21] =?UTF-8?q?=E6=9B=B4=E6=96=B0README?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> --- contrib/TSM/README.md | 141 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 contrib/TSM/README.md diff --git a/contrib/TSM/README.md b/contrib/TSM/README.md new file mode 100644 index 000000000..cc2c0f52b --- /dev/null +++ b/contrib/TSM/README.md @@ -0,0 +1,141 @@ +# TSM视频分类参考设计 + +## 1 介绍 + +### 1.1 简介 +使用TSM模型，在MindX SDK环境下实现视频分类功能。将测试视频传入脚本进行前处理，模型推理，后处理等功能，最终得到模型推理的结果。 + +### 1.2 支持的产品 + +以昇腾Atlas310卡为主要的硬件平台 + +### 1.3 支持的版本 +环境依赖软件和版本如下表： + +| 软件名称 | 版本 | +|----------|--------| +| cmake | 3.5+ | +| mxVision | 5.0.RC3 | +| Python | 3.9 | +| torch | 1.10.0 | +| ffmpeg | 4.2.1 | + +### 1.4 软件方案介绍 + +项目主要由离线精度测试文件，在线功能测试文件，离线单视频推理性能测试文件，模型文件，测试数据集预处理文件组成。 + +### 1.5 代码目录结构与说明 + +```text +├── TSM + ├── README.md // 所有模型相关说明 + ├── model + ├── onnx2om.sh // 转om模型脚本 + ├── onnx2om1.sh // 在线模型转om模型脚本 + ├── label + ├── kinetics_val.csv // label文件 + ├── download_data + ├── k400_extractor.sh // 解压数据集脚本 + ├── offline.png // 离线推理技术实现流程 + ├── online.png // 在线推理技术实现流程 + ├── online_infer.py // 在线推理精度脚本 + ├── offline_infer.py // 离线推理精度脚本 + ├── speed.py // 离线单视频推理NPU性能脚本 + ├── speed_gpu.py // 离线单视频推理GPU性能脚本 +``` + +### 1.6技术实现流程 + +在线推理流程： + +![离线推理流程](./online.png) + +### 1.7特性及适用场景 + +在线模型： + +本案例中的在线模型适用于26中手势识别，并可以返回识别手势的名称。 + +## 2 设置环境变量 + +- 环境变量搭建 + +在运行项目前，需要设置环境变量： + +MindSDK 环境变量： + +```Shell +. ${SDK-path}/set_env.sh +``` + +CANN 环境变量： + +```Shell +. ${ascend-toolkit-path}/set_env.sh +``` + +环境变量介绍 + +SDK-path: mxVision SDK 安装路径 + +ascend-toolkit-path: CANN 安装路径。 + +下载[ffmpeg](https://github.com/FFmpeg/FFmpeg/archive/n4.2.1.tar.gz)，解压进入并执行以下命令安装： + +```Shell +./configure --prefix=/usr/local/ffmpeg --enable-shared +make -j +make install +``` + +安装完毕后导入环境变量 + +```Shell +export PATH=/usr/local/ffmpeg/bin:$PATH +export LD_LIBRARY_PATH=/usr/local/ffmpeg/lib:$LD_LIBRARY_PATH +``` + +## 3 在线推理 + +**步骤1** 安装[视频流工具](https://gitee.com/ascend/docs-openmind/blob/master/guide/mindx/sdk/tutorials/reference_material/Live555%E7%A6%BB%E7%BA%BF%E8%A7%86%E9%A2%91%E8%BD%ACRTSP%E8%AF%B4%E6%98%8E%E6%96%87%E6%A1%A3.md) + +**步骤2** 生成视频流 + +根据提示当前只支持部分视频格式，并不支持.mp4后缀的文件，但可以通过ffmpeg转换生成[ffmpeg安装教程](https://gitee.com/ascend/docs-openmind/blob/master/guide/mindx/sdk/tutorials/reference_material/pc%E7%AB%AFffmpeg%E5%AE%89%E8%A3%85%E6%95%99%E7%A8%8B.md)，如下所示为MP4转换为h.264命令： + +使用ffmpeg工具将带有手势的“jester.mp4”的mp4格式视频转换生成为“jester.264”的264格式视频： + +```shell +ffmpeg -i jester.mp4 -vcodec h264 -bf 0 -g 25 -r 10 -s 1280*720 -an -f h264 jester.264 + +//-bf B帧数目控制，-g 关键帧间隔控制，-s 分辨率控制 -an关闭音频， -r 指定帧率 +``` + +使用live555生成视频流。 + +**步骤3** 模型转换 + +下载[在线模型](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/TSM/online_models.zip) jester.onnx + +将下载好的模型放在参考设计代码根目录的“model”目录下。 + +将模型转换为om模型，在“model”目录下，运行脚本生成om模型 + +```shell +bash onnx2om1.sh +``` + +**步骤4** 程序测试 + +```shell +python3 online_infer.py +``` + +修改参数，'ip:port/jester.264'为测试视频流，其中ip为起流的机器ip地址，port为起流的机器端口地址，jester.264为测试视频jester.mp4通过ffmpeg转换后的视频。 + +```python +def video2img(): + cmd = 'ffmpeg -i \"{}\" -threads 1 -vf scale=-1:331 -q:v 0 \"{}/img_%05d.jpg\"'.format('rtsp://ip:port/jester.264', './image') + subprocess.call(cmd, shell=True, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) +``` -- Gitee From 315bd7775c74e898fb56a0a5ebb155b5fa7efebf Mon Sep 17 00:00:00 2001 From: ComistryMo <12393292+comistrymo@user.noreply.gitee.com> Date: Thu, 17 Oct 2024 01:32:17 +0000 Subject: [PATCH 21/21] =?UTF-8?q?=E5=88=A0=E9=99=A4=E6=96=87=E4=BB=B6=20co?= =?UTF-8?q?ntrib/VCOD=5FSLTNet/README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- contrib/VCOD_SLTNet/README.md | 272 ---------------------------------- 1 file changed, 272 deletions(-) delete mode 100644 contrib/VCOD_SLTNet/README.md diff --git a/contrib/VCOD_SLTNet/README.md b/contrib/VCOD_SLTNet/README.md deleted file mode 100644 index e7d69da0c..000000000 --- a/contrib/VCOD_SLTNet/README.md +++ /dev/null @@ -1,272 +0,0 @@ -# 视频伪装物体检测 - -## 1 介绍 - -基于 MindX SDK 实现 SLT-Net 模型的推理，在 MoCA-Mask 数据集上 Sm 达到大于 0.6。输入连续几帧伪装物体的视频序列，输出伪装物体掩膜 Mask 图。 - - -### 1.1 支持的产品 - -支持昇腾310芯片 - - -### 1.2 支持的版本 - -CANN：7.0.RC1 - -SDK：mxVision 5.0.RC3（可通过cat SDK目录下的 version.info 查看） - - -### 1.3 软件方案介绍 - - -本方案中，先通过 `torch2onnx.py` 脚本将 PyTorch 版本的伪装视频物体检测模型 SLT-Net 转换为 onnx 模型；然后通过 `inference.py` 脚本调用晟腾om模型，将输入视频帧进行图像处理，最终生成视频伪装物体的掩膜 Mask 图。 - - -### 1.4 代码目录结构与说明 - -本sample工程名称为 VCOD_SLTNet，工程目录如下图所示： - -``` -──VCOD_SLTNet - ├── flowchart.jpeg - ├── inference.py # 推理文件 - ├── torch2onnx.py # 模型转换脚本 - └── README.md -``` - - -### 1.5 技术实现流程图 - -![Flowchart](./flowchart.jpeg) - -图1 视频伪装物体检测流程图 - - -### 1.6 特性及适用场景 - -对于伪装视频数据的分割任务均适用，输入视频需要转换为图片序列输入到模型中，具体可以参考 MoCA 数据格式与目录结构（如下所示），详见 [SLT-Net](https://xueliancheng.github.io/SLT-Net-project/) 与 [MoCA 数据集主页](https://www.robots.ox.ac.uk/~vgg/data/MoCA/)。 - - -``` ---data - └── TestDataset_per_sq # 测试数据集 - ├── flower_crab_spider_1 # 不同场景 - ├── GT # Ground Truth - ├── 00000.png - ├── ..... - └── Imgs # 输入图片序列 - ├── 00000.jpg - ├── ..... - ...... - -``` - - -## 2 环境依赖 - -环境依赖软件和版本如下表： - -| 软件名称 | 版本 | -| -------- | ------ | -| MindX SDK | 5.0.RC3 | -| Python | 3.9.2 | -| CANN | 7.0RC1 | -| PyTorch | 1.12.1 | -| numpy | 1.21.5 | -| imageio | 2.22.3| -| Pillow | 9.3.0 | -| cv2 | 4.5.5 | -| timm | 0.4.12 | -| tqdm | 4.64.1 | - - -## 3. 数据准备 - -### 3.1 准备相关文件 - -1、SLT-Net代码包准备 - -点击访问 [SLT-Net](https://github.com/XuelianCheng/SLT-Net) 并下载 SLT-Net-master.zip 代码压缩包，上传服务器并解压得到“SLT-Net-master”目录及文件； - -2、SLT-Net模型文件准备 - -方法一：通过访问 [SLT-Net 模型官方链接](https://drive.google.com/file/d/1_u4dEdxM4AKuuh6EcWHAlo8EtR7e8q5v/view) 下载模型压缩包 (注意，需要访问 Google Drive )，解压后将 `Net_epoch_MoCA_short_term_pseudo.pth` 模型拷贝至 `SLT-Net-master` 目录下； - -方法二：下载 [models.zip 备份模型压缩包](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/sltnet/models.zip) 并解压获得 `sltnet.pth`、`sltnet.onnx`、`sltnet.om` 三个模型文件，将 `sltnet.pth` 模型拷贝至 `SLT-Net-master` 目录下 - - -3、数据集准备 - -通过访问[MoCA官方链接](https://xueliancheng.github.io/SLT-Net-project/)下载 `MoCA_Video` 数据集，或者通过[数据集备份链接](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/sltnet/MoCA_Video.zip)来下载 `MoCA_Video.zip` 数据集压缩包并解压； - - -### 3.2 模型转换 - -1、SLT-Net代码预处理 - -进入 `SLT-Net-master/lib` 目录下，对 `__init__.py`、`short_term_model.py`、`pvtv2_afterTEM.py`三个文件做以下修改： - -1）`__init__.py`文件注释如下： - -``` -from .short_term_model import VideoModel as VideoModel_pvtv2 -# from .long_term_model import VideoModel as VideoModel_long_term -``` - -注：因为长期模型依赖 CUDA，并且需要在 CUDA 平台进行编译，而本项目基于 MindX SDK 实现，因此使用短期模型。并且，短期模型的评价指标满足预期。 - - -2）修改 `short_term_model.py` 文件中，如下代码行： - -修改 - -``` -def forward(self, x): - image1, image2, image3 = x[:, :3], x[:, 3:6], x[:, 6:] # 替换之前的 image1, image2, image3 = x[0],x[1],x[2] - fmap1=self.backbone.feat_net(image1) - fmap2=self.backbone.feat_net(image2) - fmap3=self.backbone.feat_net(image3) -``` - -修改 - -``` - def __init__(self, args): - super(ImageModel, self).__init__() - self.args = args - # self.backbone = Network(pvtv2_pretrained=self.args.pvtv2_pretrained, imgsize=self.args.trainsize) - self.backbone = Network(pvtv2_pretrained=self.args.pvtv2_pretrained, imgsize=352) # 指定图片大小 - - .... - - # self.backbone = Network(pvtv2_pretrained=False, imgsize=self.args.trainsize) - self.backbone = Network(pvtv2_pretrained=False, imgsize=352) # 指定图片大小 - if self.args.pretrained_cod10k is not None: - self.load_backbone(self.args.pretrained_cod10k ) -``` - - -删除 - -``` -if self.args.pretrained_cod10k is not None: - self.load_backbone(self.args.pretrained_cod10k ) -``` - - -3）`pvtv2_afterTEM.py` 文件注释如下： - -``` -from timm.models import create_model -#from mmseg.models import build_segmentor -#from mmcv import ConfigDict -import pdb -``` - - -修改“SLT-Net-master/mypath.py”文件如下： - -``` -elif dataset == 'MoCA': - return './dataset/MoCA-Mask/' # 将此处路径修改指定为“MoCA_Video”目录的相对路径 -``` - - -可参考已经完成修改的 [SLT_Net_MindXsdk_torch](https://github.com/shuowang-ai/SLT_Net_MindXsdk_torch)，也可直接使用该项目进行下面的 onnx 模型转换操作，替代以上步骤。 - - -2、模型转换 - -步骤一、pth模型转onnx模型 - -将 `VCOD_SLTNet` 代码包中的 `torch2onnx.py` 脚本拷贝至 `SLT-Net-master` 目录下，并在 `SLT-Net-master` 目录下执行以下命令将 pth 模型转换成 onnx 模型： - -``` -python torch2onnx.py --pth_path ${pth模型文件路径} --onnx_path ./sltnet.onnx -``` - -参数说明： - -pth_path：pth模型文件名称及所在路径 - -onnx_path：生成输出的onnx模型文件 - - -注意，timm 的版本为 `0.4.12`，其他版本可能有兼容性问题。 - - -步骤二、简化onnx文件（可选操作） - -``` -python -m onnxsim --input-shape="1,9,352,352" --dynamic-input-shape sltnet.onnx sltnet_sim.onnx -``` - -步骤三、onnx模型转om模型 - -``` -atc --framework=5 --model=sltnet.onnx --output=sltnet --input_shape="image:1,9,352,352" --soc_version=Ascend310 --log=error -``` - -注意： - -1. 若想使用转换好的onnx模型或om模型，可通过下载 [models.zip备份模型压缩包](https://mindx.sdk.obs.cn-north-4.myhuaweicloud.com/mindxsdk-referenceapps%20/contrib/sltnet/models.zip) 解压获得转换好的 onnx 模型或 om 模型。 - -2. pth模型转onnx模型，onnx模型转om模型，均可能花费约1小时左右，视不同运行环境而定。如无报错，请耐心等待。 - - -## 4. 运行推理 - - -使用如下命令，运行 `inference.py` 脚本： - -``` -python inference.py --datapath ${MoCA_Video数据集路径} --save_root ./results/ --om_path ./sltnet.om --testsize 352 --device_id 0 -``` - -参数说明： - -datapath：下载数据以后，目录中 `TestDataset_per_sq` 的上一级目录， - -save_root：结果保存路径 - -om_path：om 模型路径 - -testsize：图片 resize 的大小，当前固定为 352 - -device_id：设备编号 - - -注意，该脚本无需放入修改的 SLT-Net 目录，在任意位置均可执行，只需设置好上述参数即可。 - - -运行输出如下： - -``` - 0%| | 0/713 [00:00 ./results/arctic_fox/Pred/00000.png - 0%|▏ | 1/713 [00:00<10:31, 1.13it/s]> ./results/arctic_fox/Pred/00005.png - 0%|▎ | 2/713 [00:01<09:01, 1.31it/s]> ./results/arctic_fox/Pred/00010.png - 0%|▍ | 3/713 [00:02<08:30, 1.39it/s]> ./results/arctic_fox/Pred/00015.png - 1%|▌ | 4/713 [00:02<08:13, 1.44it/s]> ./results/arctic_fox/Pred/00020.png -``` - -将展示剩余运行时间以及生成图片的路径。 - - -## 5. 精度评估 - -点击访问 [SLT_Net_MindXsdk_torch](https://github.com/shuowang-ai/SLT_Net_MindXsdk_torch) 并下载 `SLT_Net_MindXsdk_torch-master.zip` 代码压缩包，上传服务器并解压获得 `SLT_Net_MindXsdk_torch-master` 目录及相关文件； - -进入 `SLT_Net_MindXsdk_torch-master` 目录，修改 `eval_python/run_eval.py` 脚本中的 `gt_dir` 为本地的 `MoCA_Video/TestDataset_per_sq/` 目录的绝对路径，`pred_dir` 为预测结果目录的绝对路径，并执行以下命令进行精度评估： - -``` -python eval_python/run_eval.py -``` - -完成评估后的结果如下： - -{'Smeasure': 0.6539, 'wFmeasure': 0.3245, 'MAE': 0.0161, 'adpEm': 0.6329, 'meanEm': 0.7229, 'maxEm': 0.7554, 'adpFm': 0.3025, 'meanFm': 0.3577, 'maxFm': 0.3738} - -评测结果高于交付所要求的 Smeasure 0.6 的指标。 - -注：评估还可参考基于 基于 [MATLAB](https://github.com/XuelianCheng/SLT-Net/tree/master/eval) 的 SLT-Net 的评测代码或参考基于 Python 的 [PySODEvalToolkit](https://github.com/lartpang/PySODEvalToolkit) 的评测代码。 -- Gitee