1 Star 0 Fork 35

CpDec/FastDeploy

forked from PaddlePaddle/FastDeploy 
Create your Gitee Account
Explore and code with more than 13.5 million developers,Free private repositories !:)
Sign up
文件
Clone or Download
contribute
Sync branch
Cancel
Notice: Creating folder will generate an empty file .keep, because not support in Git
Loading...
README

简体中文 | English

FastDeploy Serving Deployment

Introduction

FastDeploy builds an end-to-end serving deployment based on Triton Inference Server. The underlying backend uses the FastDeploy high-performance Runtime module and integrates the FastDeploy pre- and post-processing modules to achieve end-to-end serving deployment. It can achieve fast deployment with easy-to-use process and excellent performance.

Prepare the environment

Environment requirements

  • Linux
  • If using a GPU image, NVIDIA Driver >= 470 is required (for older Tesla architecture GPUs, such as T4, the NVIDIA Driver can be 418.40+, 440.33+, 450.51+, 460.27+)

Obtain Image

CPU Image

CPU images only support Paddle/ONNX models for serving deployment on CPUs, and supported inference backends include OpenVINO, Paddle Inference, and ONNX Runtime

docker pull paddlepaddle/fastdeploy:1.0.0-cpu-only-21.10

GPU Image

GPU images support Paddle/ONNX models for serving deployment on GPU and CPU, and supported inference backends including OpenVINO, TensorRT, Paddle Inference, and ONNX Runtime

docker pull paddlepaddle/fastdeploy:1.0.0-gpu-cuda11.4-trt8.4-21.10

Users can also compile the image by themselves according to their own needs, referring to the following documents:

Other Tutorials

Serving Deployment Demo

Task Model
Classification PaddleClas
Detection PaddleDetection
Detection ultralytics/YOLOv5
NLP PaddleNLP/ERNIE-3.0
NLP PaddleNLP/UIE
Speech PaddleSpeech/PP-TTS
OCR PaddleOCR/PP-OCRv3
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
Python
1
https://gitee.com/martinfields/FastDeploy.git
git@gitee.com:martinfields/FastDeploy.git
martinfields
FastDeploy
FastDeploy
develop

Search