# m1000_gpu_model_zoo

**Repository Path**: MooreThreads-AI-SOC/m1000_gpu_model_zoo

## Basic Information

- **Project Name**: m1000_gpu_model_zoo
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 4
- **Forks**: 2
- **Created**: 2025-09-26
- **Last Updated**: 2025-12-30

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# MTGPU Model Zoo

> MTGPU AI 模型集合, 本项目旨在演示如何利用 MTGPU 进行模型推理加速，帮助开发者在 MTGPU 上快速落地各种 AI 模型推理服务。

## 安装
⚠️ 请确保当前硬件环境为**E300_AI模组+T035底板(官方开发者套件)**，使用的系统软件镜像版本为**1.3.0 Release或更高版本**.
* 安装依赖：
   ```bash
   sudo apt-get install -y git cmake
   ```
* 创建 conda 环境：
  ```bash
  # 安装 conda:
  wget https://github.com/conda-forge/miniforge/releases/download/25.3.1-0/Miniforge3-25.3.1-0-Linux-aarch64.sh -O Miniforge3-25.3.1-0-Linux-aarch64.sh
  bash Miniforge3-25.3.1-0-Linux-aarch64.sh
  # 创建 conda 环境:
  conda create -n py310 python=3.10
  conda activate py310
  ```
* 设置 `pip` 镜像源：
   ```bash
   # 以清华源为例：
   pip config set install.trusted-host pypi.tuna.tsinghua.edu.cn
   ```
* 安装 MUSA SDK: [Install MUSA](./docs/install-musa.md)  
 （***注意***：如果前面重启过系统，请记得先再次进入conda环境`conda activate py310`）

* 安装 mtgpu_model_zoo:
   ```bash
   git clone --recursive https://gitee.com/MooreThreads-AI-SOC/m1000_gpu_model_zoo.git mtgpu_model_zoo
   cd mtgpu_model_zoo

   # install transformers:
   cd pkgs/transformers
   # apply transformers patch for compat with torch 2.5 `load`:
   git apply ../../patches/transformers.patch
   pip install -e .
   cd -

   # install onnx2torch
   cd pkgs/onnx2torch
   pip install -e .
   cd -

   # install ultralytics (for YOLO):
   cd pkgs/ultralytics
   git fetch --tags
   git checkout v8.3.161
   # apply the YOLO aot fix patch:
   git apply ../../mtgpu_model_zoo/models/yolo/yolo_aot_fix.patch
   pip install -e .
   cd -

   # install Cutie:
   cd pkgs/Cutie && \
      pip install -e . && cd -

   # install mtgpu_model_zoo:
   pip install -r requirements.txt
   pip install -e .

   # add your huggingface token from https://huggingface.co/docs/hub/security-tokens:
   printf "
   export HF_TOKEN=\"<your_hf_token>\"
   " >> .env.local
   ```

* 安装 TorchMUSA: [Install TorchMUSA](./docs/install-torchmusa.md)

* 运行模型推理并 perf 性能:
   ```bash
   source env.sh
   # 以 yolov8m 为例：
   python -m mtgpu_model_zoo.perf_aot -m yolov8m --use_aot
   # 您可以使用 "--help" 查看所有支持的模型：
   python -m mtgpu_model_zoo.perf_aot --help
   ```

## 项目结构

```
mtgpu-model-zoo
├── assets              # static asset files like images, model weights .etc
│   ├── images
│   └── models
├── benchmarks          # benchmarks data and scripts
│   ├── products
│   └── scripts
├── docs                # documents
├── env.sh
├── LICENSE
├── mtgpu_model_zoo     # the mtgpu_model_zoo pkg showing how to run models on GPU
│   ├── configs.py
│   ├── __init__.py
│   ├── main.py
│   ├── models
│   ├── perf_aot.py
│   ├── torch
│   └── utils
├── README.md
├── requirements.txt
├── samples
├── setup.py
├── tests               # tests
```

## 模型支持

### 图像分类 Image Classification

| Model | Format | Status | Model Link | Description |
|-------|--------|:------:|:----------:|-------------|
| [VIT](./mtgpu_model_zoo/models/vit) | PyTorch | ✅ | [🔗](https://github.com/google-research/vision_transformer) | Vision Transformer from Google |
| [NSFW](./mtgpu_model_zoo/models/nsfw) | PyTorch | ✅ | [🔗](https://huggingface.co/Falconsai/nsfw_image_detection) | NSFW (Not Safe for Work, base on ViT) |
| [Gender Classification](./mtgpu_model_zoo/models/gender_classification) | PyTorch | ✅ | [🔗](https://huggingface.co/rizvandwiki/gender-classification) | Gender classification (based on vit) |
| [SWIN](./mtgpu_model_zoo/models/swin) | PyTorch | ✅ | [🔗](https://huggingface.co/timm/swin_s3_tiny_224.ms_in1k) | classification / feature backbone (vit) |
| [EfficientNet](./mtgpu_model_zoo/models/efficientnet) | PyTorch | ✅ | [🔗](https://huggingface.co/google/efficientnet-b2) | Mobile friendly pure convolutional model (ConvNet) |
| [DINO](./mtgpu_model_zoo/models/dino) | PyTorch | ✅ | [🔗](https://huggingface.co/facebook/dinov2-base-imagenet1k-1-layer) | VIT trained using DINOv2 |
| [CLIP](./mtgpu_model_zoo/models/clip) | PyTorch | ✅ | [🔗](https://huggingface.co/openai/clip-vit-large-patch14) | CLIP model from OpenAI |
| [Chinese-CLIP-ViT](./mtgpu_model_zoo/models/cn_clip) | PyTorch | ✅ | [🔗](https://huggingface.co/OFA-Sys/chinese-clip-vit-base-patch16) | Chinese CLIP |
| [AlexNet](./mtgpu_model_zoo/models/alexnet) | ONNX | ✅ | [🔗](https://huggingface.co/Kalray/alexnet) | ImageNet Classification |
| [LeNet](./mtgpu_model_zoo/models/lenet) | ONNX | ✅ | [🔗](https://huggingface.co/EclipseAidge/LeNet) | Image Classification |
| [MobileNet-V1](./mtgpu_model_zoo/models/mobilenet_v1) | PyTorch | ✅ | [🔗](https://huggingface.co/google/mobilenet_v1_1.0_224) | Edge models for classification, detection, embeddings and segmentation |
| [MobileNet-V3](./mtgpu_model_zoo/models/mobilenet_v3) | PyTorch | ✅ | [🔗](https://huggingface.co/timm/mobilenetv3_small_100.lamb_in1k) | MobileNet V3 |
| [VGG](./mtgpu_model_zoo/models/vgg) | PyTorch | ✅ | [🔗](https://huggingface.co/timm/vgg16.tv_in1k) | Image classification / feature backbone |

### 目标检测 Object Detection

| Model | Format | Status | Model Link | Description |
|-------|--------|:------:|:----------:|-------------|
| [yolov5](./mtgpu_model_zoo/models/yolo) | PyTorch | ✅ | [🔗](https://github.com/ultralytics/ultralytics) | YOLO from Ultralytics |
| [yolov8](./mtgpu_model_zoo/models/yolo) | PyTorch | ✅ | [🔗](https://github.com/ultralytics/ultralytics) | YOLO from Ultralytics |
| [yolov10](./mtgpu_model_zoo/models/yolo) | PyTorch | ✅ | [🔗](https://github.com/ultralytics/ultralytics) | YOLO from Ultralytics |
| [yolov11](./mtgpu_model_zoo/models/yolo) | PyTorch | ✅ | [🔗](https://github.com/ultralytics/ultralytics) | YOLO from Ultralytics |
| [ADAFACE](./mtgpu_model_zoo/models/adaface) | PyTorch | ✅ | [🔗](https://github.com/mk-minchul/CVLface) | CV model for Face Recognition |
| [EfficientDet-D0](./mtgpu_model_zoo/models/efficientdet_d0) | PyTorch | ✅ | [🔗](https://huggingface.co/Kalray/efficientdet-d0) | Scalable and Efficient Object Detection |
| [FasterRCNN](./mtgpu_model_zoo/models/faster_rcnn) | ONNX | 🚧 | [🔗](https://huggingface.co/Kalray/faster-rcnn-rn50) | Real-Time Object Detection |
| [RetinaFace](./mtgpu_model_zoo/models/RetinaFace) | PyTorch | ✅ | [🔗](https://github.com/biubug6/Pytorch_Retinaface) | Single-stage Dense Face Localisation in the Wild |
| [SSD MobileNet-V2](./mtgpu_model_zoo/models/ssd_mobilnet_v2) | ONNX | ✅ | [🔗](https://huggingface.co/Kalray/ssd-mobilenet-v2) | Speed/accuracy trade-offs for modern convolutional object detectors |
| [DETR ResNet50](./mtgpu_model_zoo/models/detr_resnet50) | PyTorch | ✅ | [🔗](https://huggingface.co/facebook/detr-resnet-50) | Encoder-decoder transformer with a convolutional backbone |

### 分割模型 Segment Models

| Model | Format | Status | Model Link | Description |
|-------|--------|:------:|:----------:|-------------|
| [SAM2](./mtgpu_model_zoo/models/sam2) | PyTorch | ✅ | [🔗](https://github.com/facebookresearch/sam2) | Segment Anything Model (transformers) |
| [DeepLabV3+](./mtgpu_model_zoo/models/deeplab_v3) | PyTorch | ✅ | [🔗](https://huggingface.co/google/deeplabv3_mobilenet_v2_1.0_513) | MobileNetV2 with DeepLabV3+ |
| [FastSAM](./mtgpu_model_zoo/models/fast_sam) | ONNX | ✅ | [🔗](https://github.com/cqu20160901/FastSAM_onnx_rknn) | Fast Segment Anything Model |

### 其它模型 Other Models

| Model | Format | Status | Model Link | Description |
|-------|--------|:------:|:----------:|-------------|
| [MediaPipe-Hand](./mtgpu_model_zoo/models/mediapipe_hand_detection) | ONNX | ✅ | [🔗](https://huggingface.co/qualcomm/MediaPipe-Hand-Detection) | Real-time hand detections |
| [PP-OCRv5](./mtgpu_model_zoo/models/pp_ocr) | PaddlePaddle | ✅ | [🔗](https://huggingface.co/PaddlePaddle/PP-OCRv5_server_det) | Text detection models by the PaddleOCR team |
| [Unet-Segment](./mtgpu_model_zoo/models/unet_seg) | ONNX | ✅ | [🔗](https://huggingface.co/qualcomm/Unet-Segmentation) | Real-time segmentation |
| [SuperPoint](./mtgpu_model_zoo/models/superpoint) | PyTorch | ✅ | [🔗](https://github.com/magicleap/SuperPointPretrainedNetwork) | Interest Point Detection and Description |
| [Mask2Former](./mtgpu_model_zoo/models/Mask2former) | PyTorch | ✅ | [🔗](https://huggingface.co/facebook/mask2former-swin-base-IN21k-cityscapes-instance) | Mask Transformer for Universal Image Segmentation |
| [Cutie](./mtgpu_model_zoo/models/cutie) | PyTorch | 🚧 | [🔗](https://github.com/hkchengrex/Cutie) | Video Object Segmentation |
| [GTE Base](./mtgpu_model_zoo/models/gte_base) | PyTorch | 🚧 | [🔗](https://modelscope.cn/models/iic/gte_sentence-embedding_multilingual-base) | GTE 多语言通用文本表示模型 |
| [InsightFace](./mtgpu_model_zoo/models/insightface) | ONNX | ✅ | [🔗](https://github.com/deepinsight/insightface) | 人脸分析 |
| [Whisper](./mtgpu_model_zoo/models/whisper) | PyTorch | 🚧 | [🔗](https://huggingface.co/openai/whisper-tiny) | Automatic speech recognition (ASR) and speech translation |
| [Zipformer](./mtgpu_model_zoo/models/zipformer) | ONNX | 🚧 | [🔗](https://github.com/airockchip/rknn_model_zoo/tree/main/examples/zipformer) | Chinese-English ASR model using k2-zipformer-streaming |
| [Wespeaker](./mtgpu_model_zoo/models/wespeaker) | ONNX | ✅ | [🔗](https://huggingface.co/Wespeaker/wespeaker-voxceleb-resnet34-LM) | Speaker embedding learning and verification |

### SD/FLUX

### VLM/VLA

## 新增模型
[Add a new model](./docs/add-new-model.md)

## 测试

测试所有模型并生成测试报告：

```bash
# use --help to see all options:
python tests/test_all.py --use_aot \
   --product_mode <product_mode> \
   --incremental # add this to only run incremental tests to avoid duplicate tests
```

## 常见问题

#### 如何在 MUSA GPU 上调用 `torch.cuda` ?

请参考[常见问题](./docs/common-qa.md)。

#### 如何设置系统性能功耗模式及 GPU 工作频率 ?
请查看[性能功耗设置](./docs/working-modes.md)。

#### 模型推理报错诊断

请参考[模型推理报错诊断](./docs/model-errors-diagnostic.md)。