Linux
x86
Java
Whole Process
Inference Application
Data Preparation
Beginner
This tutorial provides an example program for MindSpore Lite to perform inference. It demonstrates the basic process of performing inference on the device side using MindSpore Lite Java API by random inputting data, executing inference, and printing the inference result. You can quickly understand how to use the Java APIs related to inference on MindSpore Lite. In this tutorial, the randomly generated data is used as the input data to perform the inference on the MobileNetV2 model and print the output data. The code is stored in the mindspore/lite/examples/quick_start_java directory.
The MindSpore Lite inference steps are as follows:
.ms
model converted by the model conversion tool (https://www.mindspore.cn/tutorial/lite/en/r1.2/use/converter_tool.html) from the file system and import the model using the loadModel.deviceType
(device type), threadNum
(number of threads), cpuBindMode
(CPU binding mode), and enable_float16
(whether to preferentially use the float16 operator).Input Tensor
.outputting the tensor
.To view the advanced usage of MindSpore Lite, see Using Runtime to Perform Inference (Java).
Environment requirements
Build
Run the build script in the mindspore/lite/examples/quick_start_java
directory to automatically download the MindSpore Lite inference framework library and model files and build the Demo.
bash build.sh
If the MindSpore Lite inference framework fails to be downloaded, manually download the MindSpore Lite model inference framework mindspore-lite-{version}-linux-x64.tar.gz whose hardware platform is CPU and operating system is Ubuntu-x64. Decompress the package and obtain the
libmindspore-lite.so
file in thelib\jar
directory. Copylibmindspore-lite-jni.so
andlibmindspore-lite-java.jar
to themindspore/lite/examples/quick_start_java/lib
directory.If the MobileNetV2 model fails to be downloaded, manually download the model file mobilenetv2.ms and copy it to the
mindspore/lite/examples/quick_start_java/model/
directory.After manually downloading and placing the file in the specified location, you need to execute the build.sh script again to complete the compilation.
Inference
After the build, go to the mindspore/lite/examples/quick_start_java/target
directory and run the following command to experience MindSpore Lite inference on the MobileNetV2 model:
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:../lib/
java -Djava.library.path=../lib/ -classpath .:./quick_start_java.jar:../lib/mindspore-lite-java.jar com.mindspore.lite.demo.Main ../model/mobilenetv2.ms
After the execution, the following information is displayed, including the tensor name, tensor size, number of output tensors, and the first 50 pieces of data.
out tensor shape: [1,1000,] and out data: 5.4091015E-5 4.030303E-4 3.032344E-4 4.0029243E-4 2.2730739E-4 8.366581E-5 2.629827E-4 3.512394E-4 2.879536E-4 1.9557697E-4xxxxxxxxxx MindSpore Lite 1.1.0out tensor shape: [1,1000,] and out data: 5.4091015E-5 4.030303E-4 3.032344E-4 4.0029243E-4 2.2730739E-4 8.366581E-5 2.629827E-4 3.512394E-4 2.879536E-4 1.9557697E-4tensor name is:Default/Sigmoid-op204 tensor size is:2000 tensor elements num is:500output data is:3.31223e-05 1.99382e-05 3.01624e-05 0.000108345 1.19685e-05 4.25282e-06 0.00049955 0.000340809 0.00199094 0.000997094 0.00013585 1.57605e-05 4.34131e-05 1.56114e-05 0.000550819 2.9839e-05 4.70447e-06 6.91601e-06 0.000134483 2.06795e-06 4.11612e-05 2.4667e-05 7.26248e-06 2.37974e-05 0.000134513 0.00142482 0.00011707 0.000161848 0.000395011 3.01961e-05 3.95325e-05 3.12398e-06 3.57709e-05 1.36277e-06 1.01068e-05 0.000350805 5.09019e-05 0.000805241 6.60321e-05 2.13734e-05 9.88654e-05 2.1991e-06 3.24065e-05 3.9479e-05 4.45178e-05 0.00205024 0.000780899 2.0633e-05 1.89997e-05 0.00197261 0.000259391
Read the MindSpore Lite model from the file system and use the model.loadModel
function to import the model for parsing.
boolean ret = model.loadModel(modelPath);
if (!ret) {
System.err.println("Load model failed, model path is " + modelPath);
return;
}
Model build includes context configuration creation, session creation, and graph build.
private static boolean compile() {
MSConfig msConfig = new MSConfig();
// You can set config through Init Api or use the default parameters directly.
// The default parameter is that the backend type is DeviceType.DT_CPU, and the number of threads is 2.
boolean ret = msConfig.init(DeviceType.DT_CPU, 2);
if (!ret) {
System.err.println("Init context failed");
return false;
}
// Create the MindSpore lite session.
session = new LiteSession();
ret = session.init(msConfig);
msConfig.free();
if (!ret) {
System.err.println("Create session failed");
model.free();
return false;
}
// Compile graph.
ret = session.compileGraph(model);
if (!ret) {
System.err.println("Compile graph failed");
model.free();
return false;
}
return true;
}
Model inference includes data input, inference execution, and output obtaining. In this example, the input data is randomly generated, and the output result is printed after inference.
private static boolean run() {
MSTensor inputTensor = session.getInputsByTensorName("2031_2030_1_construct_wrapper:x");
if (inputTensor.getDataType() != DataType.kNumberTypeFloat32) {
System.err.println("Input tensor shape do not float, the data type is " + inputTensor.getDataType());
return false;
}
// Generator Random Data.
int elementNums = inputTensor.elementsNum();
float[] randomData = generateArray(elementNums);
byte[] inputData = floatArrayToByteArray(randomData);
// Set Input Data.
inputTensor.setData(inputData);
// Run Inference.
boolean ret = session.runGraph();
if (!ret) {
System.err.println("MindSpore Lite run failed.");
return false;
}
// Get Output Tensor Data.
MSTensor outTensor = session.getOutputByTensorName("Default/head-MobileNetV2Head/Softmax-op204");
// Print out Tensor Data.
StringBuilder msgSb = new StringBuilder();
msgSb.append("out tensor shape: [");
int[] shape = outTensor.getShape();
for (int dim : shape) {
msgSb.append(dim).append(",");
}
msgSb.append("]");
if (outTensor.getDataType() != DataType.kNumberTypeFloat32) {
System.err.println("output tensor shape do not float, the data type is " + outTensor.getDataType());
return false;
}
float[] result = outTensor.getFloatData();
if (result == null) {
System.err.println("decodeBytes return null");
return false;
}
msgSb.append(" and out data:");
for (int i = 0; i < 10 && i < outTensor.elementsNum(); i++) {
msgSb.append(" ").append(result[i]);
}
System.out.println(msgSb.toString());
return true;
}
If the MindSpore Lite inference framework is not required, release the created LiteSession
and Model
.
// Delete session buffer.
session.free();
// Delete model buffer.
model.free();
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。