1 Star 0 Fork 35

ybot/FastDeploy

forked from PaddlePaddle/FastDeploy 
加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
文件
克隆/下载
compile-en.md 2.40 KB
一键复制 编辑 原始数据 按行查看 历史

English | 中文

FastDeploy Serving Deployment Compilation

Compilation with Docker containers

GPU Image

The GPU images published by FastDploy are based on version 21.10 of Triton Inference Server. If developers need to use other CUDA versions, please refer to NVIDIA official website to modify the scripts in Dockerfile and scripts.

# Enter the serving directory and execute the script to compile the FastDeploy and serving backend
cd serving
bash scripts/build.sh

# Exit to the FastDeploy home directory and create the image
# x.y.z is FastDeploy version, example: 1.0.6
cd ../
docker build -t paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.4-21.10 -f serving/Dockerfile .

The default version of TensorRT is 8.5.2.2. If you need to change the version, you can run the following commands.

cd serving
bash scripts/build.sh -tv 8.4.1.5

cd ../
docker build -t paddlepaddle/fastdeploy:x.y.z-gpu-cuda11.4-trt8.5-21.10 -f serving/Dockerfile_CUDA_11_4_TRT_8_4 .

For example, create an GPU image based on FastDeploy v1.0.6 and ubuntu 20.04,cuda11.2 environment

# Enter the serving directory and execute the script to compile the FastDeploy and serving backend
cd serving
bash scripts/build_fd_cuda_11_2.sh

# Exit to the FastDeploy home directory and create the image
cd ../
docker build -t paddlepaddle/fastdeploy:1.0.6-gpu-cuda11.2-trt8.4-21.10 -f serving/Dockerfile_CUDA_11_2 .

CPU Image

# Enter the serving directory and execute the script to compile the FastDeploy and serving backend
cd serving
cd serving
bash scripts/build.sh OFF

# Exit to the FastDeploy home directory and create the image
# x.y.z is FastDeploy version, example: 1.0.6
cd ../
docker build -t paddlepaddle/fastdeploy:x.y.z-cpu-only-21.10 -f serving/Dockerfile_cpu .

IPU Image

# Enter the serving directory and execute the script to compile the FastDeploy and serving backend
cd serving
bash scripts/build_fd_ipu.sh

# Exit to the FastDeploy home directory and create the image
# x.y.z is FastDeploy version, example: 1.0.6
cd ../
docker build -t paddlepaddle/fastdeploy:x.y.z-ipu-only-21.10 -f serving/Dockerfile_ipu .

Compilation without Docker containers

Loading...
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
Python
1
https://gitee.com/ybot/FastDeploy.git
git@gitee.com:ybot/FastDeploy.git
ybot
FastDeploy
FastDeploy
develop

搜索帮助