# xtai-benchmark **Repository Path**: xuantie-yocto/xtai-benchmark ## Basic Information - **Project Name**: xtai-benchmark - **Description**: XTAI benchmark - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2024-11-21 - **Last Updated**: 2026-04-13 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # Benchmark for XuanTie CPU Welcome to the XuanTie AI Benchmark Suite! This repository contains a comprehensive collection of benchmark tests specifically designed for RISC-V CPUs with RVV/RVM. Our goal is to provide developers, researchers, and enthusiasts with a suite of tools to evaluate the performance and efficiency of AI algorithms running on RISC-V architecture. ## Features | Model | Original Model | License | quantization | c907 | c908 | | :----------- | ------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- | --------------- | ---- | ---- | | bert | [url](https://github.com/google-research/bert) | Apache-2.0 license | float32 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float32_onnx_bert_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float32_onnx_bert_hhb.bm.tar.gz) | | | | | float16 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float16_onnx_bert_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_onnx_bert_hhb.bm.tar.gz) | | | | | int8_asym_w_sym | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_int8_asym_w_sym_onnx_bert_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_int8_asym_w_sym_onnx_bert_hhb.bm.tar.gz) | | | | | float16_w_int8 | | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_w_int8_onnx_bert_hhb.bm.tar.gz) | | efficientnet | [url](https://github.com/rwightman/gen-efficientnet-pytorch#exporting) | Apache-2.0 license | float32 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float32_onnx_efficientnet_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float32_onnx_efficientnet_hhb.bm.tar.gz) | | | | | float16 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float16_onnx_efficientnet_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_onnx_efficientnet_hhb.bm.tar.gz) | | | | | int8_asym_w_sym | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_int8_asym_w_sym_onnx_efficientnet_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_int8_asym_w_sym_onnx_efficientnet_hhb.bm.tar.gz) | | | | | float16_w_int8 | | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_w_int8_onnx_efficientnet_hhb.bm.tar.gz) | | mobilenetv2 | [url](https://github.com/onnx/models/blob/main/Computer_Vision/mobilenetv2_050_Opset16_timm/mobilenetv2_050_Opset16.onnx) | Apache-2.0 license | float32 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float32_onnx_mobilenetv2-12_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float32_onnx_mobilenetv2-12_hhb.bm.tar.gz) | | | | | float16 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float16_onnx_mobilenetv2-12_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_onnx_mobilenetv2-12_hhb.bm.tar.gz) | | | | | int8_asym_w_sym | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_int8_asym_w_sym_onnx_mobilenetv2-12_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_int8_asym_w_sym_onnx_mobilenetv2-12_hhb.bm.tar.gz) | | | | | float16_w_int8 | | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_w_int8_onnx_mobilenetv2-12_hhb.bm.tar.gz) | | mobileVit | [url](https://github.com/apple/ml-cvnets) | [apple](https://github.com/apple/ml-cvnets/blob/main/LICENSE) | float32 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float32_onnx_mobilenetVit_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float32_onnx_mobilenetVit_hhb.bm.tar.gz) | | | | | float16 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float16_onnx_mobilenetVit_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_onnx_mobilenetVit_hhb.bm.tar.gz) | | | | | int8_asym_w_sym | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_int8_asym_w_sym_onnx_mobilenetVit_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_int8_asym_w_sym_onnx_mobilenetVit_hhb.bm.tar.gz) | | | | | float16_w_int8 | | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_w_int8_onnx_mobilenetVit_hhb.bm.tar.gz) | | resnet50 | [url](https://github.com/onnx/models/blob/main/Computer_Vision/resnest50d_Opset16_timm/resnest50d_Opset16.onnx) | Apache-2.0 license | float32 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float32_onnx_resnet50-v1-7_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float32_onnx_resnet50-v1-7_hhb.bm.tar.gz) | | | | | float16 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float16_onnx_resnet50-v1-7_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_onnx_resnet50-v1-7_hhb.bm.tar.gz) | | | | | int8_asym_w_sym | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_int8_asym_w_sym_onnx_resnet50-v1-7_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_int8_asym_w_sym_onnx_resnet50-v1-7_hhb.bm.tar.gz) | | | | | float16_w_int8 | | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_w_int8_onnx_resnet50-v1-7_hhb.bm.tar.gz) | | retinaface | [url](https://github.com/biubug6/Pytorch_Retinaface) | MIT | float32 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float32_onnx_retinaface_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float32_onnx_retinaface_hhb.bm.tar.gz) | | | | | float16 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float16_onnx_retinaface_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_onnx_retinaface_hhb.bm.tar.gz) | | | | | int8_asym_w_sym | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_int8_asym_w_sym_onnx_retinaface_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_int8_asym_w_sym_onnx_retinaface_hhb.bm.tar.gz) | | | | | float16_w_int8 | | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_int8_asym_w_sym_onnx_retinaface_hhb.bm.tar.gz) | | shufflenetv2 | [url](https://github.com/onnx/models/tree/main/Computer_Vision) | Apache-2.0 license | float32 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float32_onnx_shufflenet-v2-9_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float32_onnx_shufflenet-v2-9_hhb.bm.tar.gz) | | | | | float16 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float16_onnx_shufflenet-v2-9_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_onnx_shufflenet-v2-9_hhb.bm.tar.gz) | | | | | int8_asym_w_sym | | | | | | | float16_w_int8 | | | | swin | [url](https://github.com/microsoft/Swin-Transformer) | MIT | float32 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float32_onnx_swin_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float32_onnx_swin_hhb.bm.tar.gz) | | | | | float16 | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_float16_onnx_swin_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_onnx_swin_hhb.bm.tar.gz) | | | | | int8_asym_w_sym | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/907_int8_asym_w_sym_onnx_swin_hhb.bm.tar.gz) | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_int8_asym_w_sym_onnx_swin_hhb.bm.tar.gz) | | | | | float16_w_int8 | | [url](http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_w_int8_onnx_swin_hhb.bm.tar.gz) | | yolov5n | [url](https://github.com/ultralytics/yolov5) | AGPL-3.0 | float32 | ✓ | ✓ | | | | | float16 | ✓ | ✓ | | | | | int8_asym_w_sym | ✓ | ✓ | | | | | float16_w_int8 | | ✓ | ## Getting Started ### Requirements - riscv gcc: https://www.xrvm.cn/community/download?id=4333581795569242112 - qemu(not necessary): https://www.xrvm.cn/community/download?id=4300747789186437120 ### Clone codes ```Shell # 1. download codes git clone repo_url # 2. prepare submodules cd REPO_DIR git submodule init git submodule update ``` ### Evaluations Take `c908/float16/onnx_mobilenetv2-12` as an example ```Shell cd c908/float16/onnx_mobilenetv2-12 make -f makefile.c908 hhb_runtime # download binary file for model(click the url in the above table or wget) wget http://yocbook.oss-cn-hangzhou.aliyuncs.com/linux_image/XuanTie_V1.0.5/xtai-benchmark-bm/908_float16_onnx_mobilenetv2-12_hhb.bm.tar.gz tar -zxvf 908_float16_onnx_mobilenetv2-12_hhb.bm.tar.gz mv 908_float16_onnx_mobilenetv2-12_hhb.bm hhb.bm # run with qemu make -f makefile.c908 run_sim ```