13 Star 75 Fork 28

PaddlePaddle / FastDeploy

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
README_CN.md 66.26 KB
一键复制 编辑 原始数据 按行查看 历史
yeliang2258 提交于 2023-04-19 18:45 . [Doc] Update doc for smoke (#1823)

English | 简体中文 | हिन्दी | 日本語 | 한국인 | Pу́сский язы́к

⚡️FastDeploy

安装 | 使用文档 | 快速开始 | API文档 | 更新日志


⚡️FastDeploy是一款全场景易用灵活极致高效的AI推理部署工具, 支持云边端部署。提供超过 🔥160+ TextVisionSpeech跨模态模型📦开箱即用的部署体验,并实现🔚端到端的推理性能优化。包括 物体检测字符识别(OCR)人脸人像扣图多目标跟踪系统NLPStable Diffusion文图生成TTS 等几十种任务场景,满足开发者多场景、多硬件、多平台的产业部署需求。

🌠 近期更新

🌌 推理后端及能力

X86_64 CPU       





NVDIA GPU




飞腾 CPU
昆仑芯 XPU
华为昇腾 NPU
Graphcore IPU
算能
Intel 显卡
Jetson




ARM CPU

RK3588等
RV1126等
晶晨
恩智浦

🔮 文档教程

快速开始💨

Python SDK快速开始(点开收缩)

🎆 快速安装

🔸 前置依赖

  • CUDA >= 11.2、cuDNN >= 8.0、Python >= 3.6
  • OS: Linux x86_64/macOS/Windows 10

🔸 安装GPU版本

pip install numpy opencv-python fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

🔸 Conda安装(推荐✨)

conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2

🔸 安装CPU版本

pip install numpy opencv-python fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

🎇 Python 推理示例

  • 准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
  • 测试推理结果
# GPU/TensorRT部署参考 examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision

model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
                                 "ppyoloe_crn_l_300e_coco/model.pdiparams",
                                 "ppyoloe_crn_l_300e_coco/infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im)
print(result)

vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
C++ SDK快速开始(点开查看详情)

🎆 安装

🎇 C++ 推理示例

  • 准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
  • 测试推理结果
// GPU/TensorRT部署参考 examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"

int main(int argc, char* argv[]) {
  namespace vision = fastdeploy::vision;
  auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
                                          "ppyoloe_crn_l_300e_coco/model.pdiparams",
                                          "ppyoloe_crn_l_300e_coco/infer_cfg.yml");
  auto im = cv::imread("000000014439.jpg");

  vision::DetectionResult res;
  model.Predict(im, &res);

  auto vis_im = vision::VisDetection(im, res, 0.5);
  cv::imwrite("vis_image.jpg", vis_im);
  return 0;
}

更多部署案例请参考模型部署示例 .

✴️ ✴️ 服务端模型支持列表 ✴️ ✴️

符号说明: (1) ✅ : 已经支持; (2) ❔: 正在进行中; (3) N/A : 暂不支持.

服务端模型支持列表(点击可收缩)
任务场景 模型 Linux Linux Win Win Mac Mac Linux Linux Linux Linux Linux Linux Linux
--- --- X86 CPU NVIDIA GPU X86 CPU NVIDIA GPU X86 CPU Arm CPU AArch64 CPU 飞腾D2000 aarch64 NVIDIA Jetson Graphcore IPU 昆仑芯 XPU 华为 昇腾 Serving
Classification PaddleClas/ResNet50
Classification TorchVison/ResNet
Classification ultralytics/YOLOv5Cls
Classification PaddleClas/PP-LCNet
Classification PaddleClas/PP-LCNetv2
Classification PaddleClas/EfficientNet
Classification PaddleClas/GhostNet
Classification PaddleClas/MobileNetV1
Classification PaddleClas/MobileNetV2
Classification PaddleClas/MobileNetV3
Classification PaddleClas/ShuffleNetV2
Classification PaddleClas/SqueeezeNetV1.1
Classification PaddleClas/Inceptionv3
Classification PaddleClas/PP-HGNet
Detection 🔥🔥PaddleDetection/PP-YOLOE+
Detection 🔥PaddleDetection/YOLOv8
Detection 🔥ultralytics/YOLOv8
Detection PaddleDetection/PicoDet
Detection PaddleDetection/YOLOX
Detection PaddleDetection/YOLOv3
Detection PaddleDetection/PP-YOLO
Detection PaddleDetection/PP-YOLOv2
Detection PaddleDetection/Faster-RCNN
Detection PaddleDetection/Mask-RCNN
Detection Megvii-BaseDetection/YOLOX
Detection WongKinYiu/YOLOv7
Detection WongKinYiu/YOLOv7end2end_trt
Detection WongKinYiu/YOLOv7end2end_ort
Detection meituan/YOLOv6
Detection ultralytics/YOLOv5
Detection WongKinYiu/YOLOR
Detection WongKinYiu/ScaledYOLOv4
Detection ppogg/YOLOv5Lite ?
Detection RangiLyu/NanoDetPlus
Perception Paddle3D/Smoke
KeyPoint PaddleDetection/TinyPose
KeyPoint PaddleDetection/PicoDet + TinyPose
HeadPose omasaht/headpose
Tracking PaddleDetection/PP-Tracking
OCR PaddleOCR/PP-OCRv2
OCR PaddleOCR/PP-OCRv3
Segmentation PaddleSeg/PP-LiteSeg
Segmentation PaddleSeg/PP-HumanSegLite
Segmentation PaddleSeg/HRNet
Segmentation PaddleSeg/PP-HumanSegServer
Segmentation PaddleSeg/Unet
Segmentation PaddleSeg/Deeplabv3
FaceDetection biubug6/RetinaFace
FaceDetection Linzaer/UltraFace
FaceDetection deepcam-cn/YOLOv5Face
FaceDetection insightface/SCRFD
FaceAlign Hsintao/PFLD
FaceAlign Single430/FaceLandmark1000
FaceAlign jhb86253817/PIPNet
FaceRecognition insightface/ArcFace
FaceRecognition insightface/CosFace
FaceRecognition insightface/PartialFC
FaceRecognition insightface/VPL
Matting ZHKKKe/MODNet
Matting PeterL1n/RobustVideoMatting
Matting PaddleSeg/PP-Matting
Matting PaddleSeg/PP-HumanMatting
Matting PaddleSeg/ModNet
Video Super-Resolution PaddleGAN/BasicVSR
Video Super-Resolution PaddleGAN/EDVR
Video Super-Resolution PaddleGAN/PP-MSVSR
Information Extraction PaddleNLP/UIE
NLP PaddleNLP/ERNIE-3.0
Speech PaddleSpeech/PP-TTS --

📳 移动端和端侧 模型支持列表

端侧模型支持列表(点击可收缩)
任务场景 模型 大小(MB) Linux Android Linux Linux Linux Linux Linux 更新中...
--- --- --- ARM CPU ARM CPU 瑞芯微NPU
RK3588/RK3568/RK3566
瑞芯微NPU
RV1109/RV1126/RK1808
晶晨NPU
A311D/S905D/C308X
恩智浦NPU
i.MX 8M Plus
更新中...
Classification PaddleClas/ResNet50 98
Classification PaddleClas/PP-LCNet 11.9 -- -- --
Classification PaddleClas/PP-LCNetv2 26.6 -- -- --
Classification PaddleClas/EfficientNet 31.4 -- -- --
Classification PaddleClas/GhostNet 20.8 -- -- --
Classification PaddleClas/MobileNetV1 17 -- -- --
Classification PaddleClas/MobileNetV2 14.2 -- -- --
Classification PaddleClas/MobileNetV3 22 --
Classification PaddleClas/ShuffleNetV2 9.2 -- -- --
Classification PaddleClas/SqueezeNetV1.1 5 -- -- --
Classification PaddleClas/Inceptionv3 95.5 -- -- --
Classification PaddleClas/PP-HGNet 59 -- -- --
Detection PaddleDetection/PicoDet_s 4.9 --
Detection YOLOv5 --
Face Detection deepinsight/SCRFD 2.5 -- -- -- --
Keypoint Detection PaddleDetection/PP-TinyPose 5.5 --
Segmentation PaddleSeg/PP-LiteSeg(STDC1) 32.2 -- -- -- --
Segmentation PaddleSeg/PP-HumanSeg-Lite 0.556 -- -- -- --
Segmentation PaddleSeg/HRNet-w18 38.7 -- -- -- --
Segmentation PaddleSeg/PP-HumanSeg 107.2 -- -- -- --
Segmentation PaddleSeg/Unet 53.7 -- -- -- --
Segmentation PaddleSeg/Deeplabv3 150
OCR PaddleOCR/PP-OCRv2 2.3+4.4 -- -- -- --
OCR PaddleOCR/PP-OCRv3 2.4+10.6 --

⚛️ Web和小程序 模型支持列表

Web和小程序部署支持列表(点击可收缩)
任务场景 模型 web_demo
--- --- Paddle.js
Detection FaceDetection
Detection ScrewDetection
Segmentation PaddleSeg/HumanSeg
Object Recognition GestureRecognition
Object Recognition ItemIdentification
OCR PaddleOCR/PP-OCRv3

💐 Acknowledge

本项目中SDK生成和下载使用了EasyEdge中的免费开放能力,在此表示感谢。

©️ License

FastDeploy遵循Apache-2.0开源协议

Python
1
https://gitee.com/paddlepaddle/FastDeploy.git
git@gitee.com:paddlepaddle/FastDeploy.git
paddlepaddle
FastDeploy
FastDeploy
develop

搜索帮助