代码拉取完成，页面将自动刷新
English | 简体中文
Note: The content of this SIG follows the convention described in OpenHarmony's PMC Management Charter README.
DLLite-Micro is a lightweight artificial intelligence (AI) inference framework that supports deep learning model inference on mini and small devices running OpenHarmony. DLLite-Micro provides you with clear and easy-to-use northbound APIs, helping you deploy deep learning models on the device side more easily. It can work with a variety of basic inference frameworks that are deployable on different types of underlying hardware. Currently, DLLite-Micro only adapts to the MindSpore Lite for IoT inference framework. More basic inference frameworks will be supported in the future.
DLLite-Micro is a lightweight artificial intelligence (AI) inference framework that supports deep learning model inference on mini and small devices running OpenHarmony. DLLite-Micro provides you with clear and easy-to-use northbound APIs, helping you deploy deep learning models on the device side more easily. It can work with a variety of basic inference frameworks that are deployable on different types of underlying hardware. Currently, DLLite-Micro only adapts to the MindSpore Lite for IoT inference framework. More basic inference frameworks will be supported in the future.
Figure 1 DLLite-Micro framework architecture
/foundation/ai/dllite-micro # Main directory of the DLLite-Micro framework
├── interfaces
│ └── kits
│ └── interpreter # DLLite-Micro external interfaces
├── samples # DLLite-Micro application demo
│ ├── app # Application samples
│ └── model # Model compilation samples
├── services
│ ├── inferframework # Basic inference framework adaptation module
│ ├── interpreter # Basic inference module
│ └── third_party # Third-party dependencies
│ └── mindspore_lite # MindSpore Lite dependencies
├── test # Module test
│ └── unittest # Unit test
│ └── common # Common cases
Supported language: C++
Operating system: OpenHarmony
Architecture differences: The code conversion tool provided by MindSpore Lite combines the model structure and weight on the ARM32M platform and does not generate a separate weight file. Therefore, if you are using the ARM32M platform, DLLite-Micro reads the model by combining the model structure and weight, and you need to set ModelConfig.weightSeparateFlag_ to false. If you are using the ARM32A platform, DLLite-Micro reads the model by separating the model structure from the weight, and you need to set ModelConfig.weightSeparateFlag_ to true.
Build the DLLite-Micro framework.
For the lightweight AI inference engine framework module, the source code is stored in /foundation/ai/dllite_micro/services.
The build commands are as follows:
Set the build path.
hb set -root dir // root directory of OpenHarmony
Specify a product for building. (Select a product with arrow keys and press Enter.)
hb set -p
Add DLLite-Micro component
Add the configuration of the DLLite-Micro component to the /build/lite/components/ai.json file. The sample code below shows some configurations defined in the ai.json file, and the code between ##start## and ##end## is the new configuration (Delete the rows where ##start## and ##end## are located after the configurations are added.)
{
"components": [
{
"component": "ai_engine",
"description": "AI engine framework.",
"optional": "true",
"dirs": [
"foundation/ai/engine"
],
"targets": [
"//foundation/ai/engine/services:ai"
],
"rom": "130KB",
"ram": "~337KB",
"output": [
"ai_server",
"ai_communication_adapter.a"
],
"adapted_kernel": [
"liteos_a",
"linux"
],
"features": [],
"deps": {
"third_party": [
"bounds_checking_function",
"iniparser"
],
"kernel_special": {},
"board_special": {},
"components": [
"hilog",
"utils_base",
"ipc_lite",
"samgr_lite"
]
}
},
##start##
{
"component": "ai_dllite_micro",
"description": "DLLite-micro framework.",
"optional": "true",
"dirs": [
"foundation/ai/dllite_micro"
],
"targets": [
"//foundation/ai/dllite_micro/services:ai_dllite_micro"
],
"rom": "",
"ram": "",
"output": [
"libdlliteclient.so",
"libdlliteclient_mslite_for_iot.so"
],
"adapted_kernel": ["liteos_a"],
"features": [],
"deps": {
"third_party": [],
"components": []
}
}
##end##
]
}
Modify the board configuration file.
Add the DLLite-Micro component to the vendor/hisilicon/hispark_taurus/config.json file. The sample code below shows the configurations of the ai subsystem, and the code between ##start## and ##end## is the new configuration (Delete the rows where ##start## and ##end## are located after the configurations are added.)
{
"subsystem": "ai",
"components": [
{ "component": "ai_engine", "features":[] },
##start##
{ "component": "ai_dllite_micro", "features": [] }
##end##
]
},
Start building.
hb build -f // Build the entire repository.
Or hb build dllite_micro // Build only the dllite_micro component.
Note: For details about how to set up the system environment, see OpenHarmony Quick Start. For details about how to install and use the hb utility, see the Lightweight Building Component.
Compile a model.
Currently, the framework only supports loading the inference model that is compiled into a dynamic library. You need to compile the model dynamic library in a specific mode based on the basic framework in use. The following describes the model compilation processes of different frameworks.
MindSpore 1.2 framework model compilation:
For details about how to use the MindSpore 1.2 framework for inference, see the sample provided in /foundation/ai/dllite_micro/samples/model/mnist. The following is the directory for compiling the model:
/dllite_micro/samples/model/mnist # Sample for compilation of the MNIST classification model
├── include # Header files
│ ├── nnacl # Header file of nnacl operators
│ ├── wrapper
│ └── mindspore_adapter.h
├── lib # Dependent library files
│ ├── libmindspore-lite.a # MindSpore Lite operator library
│ └── libwrapper.a # MindSpore Lite API library
├── src # Source files
│ ├── micro # Inference source code generated by the codegen tool
│ └── mindspore_adapter.cpp # MindSpore Lite adapter that provides external APIs
└── BUILD.gn # GN configuration file
Download the MindSpore Lite software package of the desired version from the MindSpore open source website.
For a model not under the MindSpore framework, you need to use the converter provided in the software package to convert the original model into an MS model. For details about how to use the converter, see Converting Models for Inference.
Use codegen to convert the MS model into the model code in C or C++ format. The model code is generated in the src directory. Copy the model code in the src directory to the /foundation/ai/dllite_micro/samples/model/mnist/src/micro directory.
Rename the model.h file in the /foundation/ai/dllite_micro/samples/model/mnist/src/micro directory as mmodel.h. In addition, change #include"model.h" in the session.cc file into #include"mmodel.h" to avoid confusion with /foundation/ai/dllite_micro/services/third_party/mindspore_lite/include/model.h.
NOTICE: Copy the net.bin model weight file in the src directory. The model weight file needs to be loaded for inference in subsequent steps.
Download the OpenHarmony software package corresponding to MindSpore r1.2, obtain the operator library and API library (inference/lib/libmindspore-lite.a and tools/lib/libwrapper.a), and copy them to the /foundation/ai/dllite_micro/samples/model/mnist/lib directory.
Copy the inference framework header files (tools/codegen/nnacl/ and tools/codegen/wrapper/) in the MindSpore Lite software package to the /foundation/ai/dllite_micro/samples/model/mnist/include directory.
Modify the /build/lite/component/ai.json file and add the model compilation configuration. The following is a code segment of the ai.json file: The content between ##start## and ##end## is the new configuration. (##start## and ##end## are only used to identify the positions. After the configuration is added, delete the two lines.)
{
"component": "ai_dllite_micro",
"description": "DLLite-Micro framework.",
"optional": "true",
"dirs": [
"foundation/ai/dllite_micro"
],
"targets": [
"//foundation/ai/dllite_micro/services:ai_dllite_micro",
##start##
"//foundation/ai/dllite_micro/samples:dllite_micro_sample_model"
##end##
],
"rom": "",
"ram": "",
"output": [
"libdlliteclient.so",
"libdlliteclient_mslite_for_iot.so"
],
"adapted_kernel": [ "liteos_a" ],
"features": [],
"deps": {
"components": [],
"third_party": []
}
},
Compile DLLite-Micro. The generated model dynamic library is stored in /usr/lib/libmnist.so.
NOTE: For details about how to download and use the MindSpore model converter and codegen tools, go to the MindSpore open source website.
Develop samples. (For details, see the MNIST classification demo.)
The /foundation/ai/dllite-micro/samples/app/mnist directory provides a sample application to call the API provided by DLLite-Micro to load the model dynamic library and model weight for inference.
Creating an instance:
static int CreatInterpreter()
{
// RegisterFeature
g_featureConfig.featureName = FEATURE_NAME;
featureInterpreter = FeatureInterpreter::RegisterFeature(featureConfig);
if (featureInterpreter.get() == nullptr) {
std::cout << "RegisterFeature failed" << std::endl;
return -1;
}
// CreateModelInterpreter
modelConfig.inferFrameworkType_ = InferFrameworkType::MINDSPORE;
modelInterpreter = featureInterpreter->CreateModelInterpreter(modelConfig);
if (modelInterpreter == nullptr) {
std::cout << "CreateModelInterpreter failed" << std::endl;
return -1;
}
return 0;
}
Calling the inference process:
static int ModelInference()
{
// Load
ReturnCode returnCode = modelInterpreter->Load();
if (returnCode != ReturnCode::SUCCESS) {
std::cout << "Load failed" << std::endl;
return -1;
}
// GetTensors
returnCode = modelInterpreter->GetTensors(inputs, IOFlag::INPUT);
if (returnCode != ReturnCode::SUCCESS) {
std::cout << "GetTensors inputs failed" << std::endl;
return -1;
}
returnCode = modelInterpreter->GetTensors(outputs, IOFlag::OUTPUT);
if (returnCode != ReturnCode::SUCCESS) {
std::cout << "GetTensors outputs failed" << std::endl;
return -1;
}
SetInputTensor(inputs);
// Invoke
WarmUp(modelInterpreter);
for (int i = 0; i < INVOKE_LOOP_COUNT; ++i) {
returnCode = modelInterpreter->Invoke(inputs, outputs);
if (returnCode != ReturnCode::SUCCESS) {
std::cout << Invoke failed" << std::endl;
return -1;
}
}
returnCode = modelInterpreter->GetTensors(outputs, IOFlag::OUTPUT);
PrintTensors(inputs, outputs);
// Unload
returnCode = modelInterpreter->Unload();
if (returnCode != ReturnCode::SUCCESS) {
std::cout << "Unload failed" << std::endl;
return -1;
}
return 0;
}
Destroying the instance:
static int DestoryInterpreter()
{
// DestroyModelInterpreter
ReturnCode returnCode = featureInterpreter->DestroyModelInterpreter(modelInterpreter);
if (returnCode != ReturnCode::SUCCESS) {
std::cout << "DestroyModelInterpreter failed" << std::endl;
return -1;
}
// UnregisterFeature
returnCode = FeatureInterpreter::UnregisterFeature(featureInterpreter);
if (returnCode != ReturnCode::SUCCESS) {
std::cout << "UnregisterFeature failed" << std::endl;
return -1;
}
return 0;
}
Compiling the sample application:
As shown below, add the dllite_micro_sample configuration to the ai.json file. (##start## and ##end## are only used to identify the positions. After the configuration is added, delete the two lines.)
{
"component": "ai_dllite_micro",
"description": "DLLite-Micro framework.",
"optional": "true",
"dirs": [
"foundation/ai/dllite_micro"
],
"targets": [
"//foundation/ai/dllite_micro/services:ai_dllite_micro",
"//foundation/ai/dllite_micro/samples:dllite_micro_sample_model",
##start##
"//foundation/ai/dllite_micro/samples:dllite_micro_sample"
##end##
],
"rom": "",
"ram": "",
"output": [
"libdlliteclient.so",
"libdlliteclient_mslite_for_iot.so"
],
"adapted_kernel": [ "liteos_a" ],
"features": [],
"deps": {
"components": [],
"third_party": []
}
},
The sample application requires the model dynamic library and model weight files. Add the following commands to /foundation/ai/dllite-micro/samples/app/mnist/BUILD.gn. During compilation, copy the model weight file generated by MindSpore Lite to the /storage/data/ directory of OpenHarmony.
copy("sample_model") {
sources = ["//foundation/ai/dllite_micro/samples/model/mnist/src/micro/net.bin"]
outputs = ["$root_out_dir/data/dllite_micro_mnist.bin"]
}
Compile DLLite-Micro. The sample application generated after compilation is stored in /bin/dllite_micro_mnist_sample.bin. Run the following command on OpenHarmony to run the application:
cd /bin
./dllite_micro_mnist_sample.bin /usr/lib/libmnist.so /storage/data/dllite_micro_mnist.bin
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。