{"release":{"tag":{"name":"v0.5.0","path":"/omniai/omniinfer/tags/v0.5.0","tree_path":"/omniai/omniinfer/tree/v0.5.0","message":"# v0.5.0\r\n\r\n## 核心特性\r\n\r\n* 支持VeRL\r\n\r\n\r\n## 其它优化\r\n* 基于5P8-1D32@A3，2K+2K，Deepseek R1性能达到QPM500，TTFT\u003CTPOT\u003C50ms\r\n\r\n## 支持模型列表\r\n\r\n| 模型| 硬件|精度类型|部署形态 |\r\n| --- | --- |--- |--- |\r\n| DeepSeek-R1| A3|INT8|PD分离 |\r\n| DeepSeek-R1| A3|W4A8C16|PD分离 |\r\n| DeepSeek-R1| A3|BF16|PD分离 |\r\n| DeepSeek-R1| A2|INT8|PD分离 |\r\n| Qwen2.5-7B |A3|INT8|混布（TP\u003E=1 DP=1） |\r\n| Qwen2.5-7B |A2|INT8|混布（TP\u003E=1 DP=1） |\r\n| QwQ |A3|BF16|PD分离 |\r\n| Qwen3-32B |A3|BF16|PD分离 |\r\n| Qwen3-235B| A3|INT8|PD分离 |\r\n| Kimi-K2| A3|W4A8C16|PD分离 |\r\n\r\n\r\n## 安装包\r\n| 硬件| 架构|镜像文件|Tar包 |\r\n| --- | --- |--- |--- |\r\n| A3| arm|docker push swr.cn-east-4.myhuaweicloud.com/omni/omni_infer-a3-arm:release_v0.5.0|[omni_infer-a3-arm:v0.5.0]( https://bucket-omni-infer-wuhu.obs.myhuaweicloud.com:443/DockerImage/Omni-Infer/Release/v0.5.0/ARM/omni_infer-a3-arm-v0.5.0.tar?AccessKeyId=HPUABVUGOTP2OPODPEKP\u0026Expires=1789647207\u0026Signature=soRfOmrTUQ2kL9hV%2BlsMhtigAgA%3D) |\r\n| A3| x86|docker pull swr.cn-east-4.myhuaweicloud.com/omni/omni_infer-a3-x86:release_v0.5.0|[omni_infer-a3-x86:v0.5.0](https://bucket-omni-infer-wuhu.obs.myhuaweicloud.com:443/DockerImage/Omni-Infer/Release/v0.5.0/X86/omni_infer-a3-x86-v0.5.0.tar?AccessKeyId=HPUABVUGOTP2OPODPEKP\u0026Expires=1789647563\u0026Signature=esbkaiPe3kdwdOYd3K2i5ilcHkY%3D)|\r\n| A2| arm|docker pull swr.cn-east-4.myhuaweicloud.com/omni/omni_infer-a2-arm:release_v0.5.0|[omni_infer-a2-arm:v0.5.0](https://bucket-omni-infer-wuhu.obs.myhuaweicloud.com:443/DockerImage/Omni-Infer/Release/v0.5.0/ARM/omni_infer-a2-arm-v0.5.0.tar?AccessKeyId=HPUABVUGOTP2OPODPEKP\u0026Expires=1789647146\u0026Signature=Ovj1kCPS6OWy/2c4SuVN1rEF0Y4%3D)|\r\n| A2| x86| docker pull swr.cn-east-4.myhuaweicloud.com/omni/omni_infer-a2-x86:release_v0.5.0|[omni_infer-a2-x86:v0.5.0](https://bucket-omni-infer-wuhu.obs.myhuaweicloud.com:443/DockerImage/Omni-Infer/Release/v0.5.0/X86/omni_infer-a2-x86-v0.5.0.tar?AccessKeyId=HPUABVUGOTP2OPODPEKP\u0026Expires=1789647335\u0026Signature=UDdizej5VIj2ubI/mej0e8UA8wc%3D)|\r","commit":{"id":"e90e8a5d6b5c2c823310dc99670f54dfad3faef0","short_id":"e90e8a5","title":"!671 add prefill config for a2 w4a8","title_markdown":"\u003Ca title=\"Pull Request: add prefill config for a2 w4a8\" class=\"gfm gfm-pull_request\" href=\"/omniai/omniinfer/pulls/671\"\u003E!671\u003C/a\u003Eadd prefill config for a2 w4a8","description":"Merge pull request !671 from release_v0.5.0-a2config","description_markdown":"Merge pull request \u003Ca title=\"Pull Request: add prefill config for a2 w4a8\" class=\"gfm gfm-pull_request\" href=\"/omniai/omniinfer/pulls/671\"\u003E!671\u003C/a\u003Efrom release_v0.5.0-a2config","message":"!671 add prefill config for a2 w4a8\nMerge pull request !671 from release_v0.5.0-a2config","message_markdown":"\u003Ca title=\"Pull Request: add prefill config for a2 w4a8\" class=\"gfm gfm-pull_request\" href=\"/omniai/omniinfer/pulls/671\"\u003E!671\u003C/a\u003Eadd prefill config for a2 w4a8\nMerge pull request \u003Ca title=\"Pull Request: add prefill config for a2 w4a8\" class=\"gfm gfm-pull_request\" href=\"/omniai/omniinfer/pulls/671\"\u003E!671\u003C/a\u003Efrom release_v0.5.0-a2config","detail_path":"/omniai/omniinfer/commit/e90e8a5d6b5c2c823310dc99670f54dfad3faef0","commits_path":"/omniai/omniinfer/commits/e90e8a5d6b5c2c823310dc99670f54dfad3faef0","tree_path":"/omniai/omniinfer/tree/e90e8a5d6b5c2c823310dc99670f54dfad3faef0","author":{"name":"liujianxin","email":"liujianxin@huawei.com","username":"octol","user_path":"/octol","enterprise_user_path":"/omniai/dashboard/members/octol","image_path":"no_portrait.png#liujianxin-octol","is_gitee_user":true,"is_enterprise_user":true,"widget_url":""},"committer":{"name":"Gitee GPG Bot","email":"noreply@gitee.com","username":"gitee-bot","user_path":"/gitee-bot","enterprise_user_path":null,"image_path":"https://foruda.gitee.com/avatar/1677201213385506226/10186697_gitee-bot_1639518846.png!avatar30","is_gitee_user":true,"is_enterprise_user":false,"widget_url":""},"authored_date":"2025-09-22T06:12:24+00:00","committed_date":"2025-09-22T06:12:24+00:00","signature":null,"build_state":null},"archive_path":"/omniai/omniinfer/repository/archive/v0.5.0","signature":null},"operating":{"edit":false,"download":true,"destroy":false,"enterprise_forbid_zip":false},"release":{"title":"Omni_infer v0.5.0 Release Note","path":"/omniai/omniinfer/releases/tag/v0.5.0","tag_path":"/omniai/omniinfer/tree/v0.5.0","project_id":41288219,"created_at":"2025-09-23T19:10:02+08:00","is_prerelease":false,"description":"# v0.5.0\r\n\r\n## 核心特性\r\n\r\n* 支持VeRL\r\n\r\n\r\n## 其它优化\r\n* 基于5P8-1D32@A3，平均3.5K+1K，Deepseek R1性能达到QPM500，TTFT\u003C2s，TPOT\u003C50ms\r\n* 基于1P16-1D32@A2，2K+2K，Deepseek R1单卡Decode峰值性能达到400 TPS，TPOT\u003C50ms\r\n\r\n## 支持模型列表\r\n\r\n| 模型| 硬件|精度类型|部署形态 |\r\n| --- | --- |--- |--- |\r\n| DeepSeek-R1| A3|INT8|PD分离 |\r\n| DeepSeek-R1| A3|W4A8C16|PD分离 |\r\n| DeepSeek-R1| A3|BF16|PD分离 |\r\n| DeepSeek-R1| A2|INT8|PD分离 |\r\n| Qwen2.5-7B |A3|INT8|混布（TP\u003E=1 DP=1） |\r\n| Qwen2.5-7B |A2|INT8|混布（TP\u003E=1 DP=1） |\r\n| QwQ |A3|BF16|PD分离 |\r\n| Qwen3-32B |A3|BF16|PD分离 |\r\n| Qwen3-235B| A3|INT8|PD分离 |\r\n| Kimi-K2| A3|W4A8C16|PD分离 |\r\n\r\n\r\n## 安装包\r\n| 硬件| 架构|镜像文件|Tar包 |\r\n| --- | --- |--- |--- |\r\n| A3| arm|docker pull swr.cn-east-4.myhuaweicloud.com/omni/omni_infer-a3-arm:release_v0.5.0|[omni_infer-a3-arm:v0.5.0]( https://bucket-omni-infer-wuhu.obs.myhuaweicloud.com:443/DockerImage/Omni-Infer/Release/v0.5.0/ARM/omni_infer-a3-arm-v0.5.0.tar?AccessKeyId=HPUABVUGOTP2OPODPEKP\u0026Expires=1789647207\u0026Signature=soRfOmrTUQ2kL9hV%2BlsMhtigAgA%3D) |\r\n| A3| x86|docker pull swr.cn-east-4.myhuaweicloud.com/omni/omni_infer-a3-x86:release_v0.5.0|[omni_infer-a3-x86:v0.5.0](https://bucket-omni-infer-wuhu.obs.myhuaweicloud.com:443/DockerImage/Omni-Infer/Release/v0.5.0/X86/omni_infer-a3-x86-v0.5.0.tar?AccessKeyId=HPUABVUGOTP2OPODPEKP\u0026Expires=1789647563\u0026Signature=esbkaiPe3kdwdOYd3K2i5ilcHkY%3D)|\r\n| A2| arm|docker pull swr.cn-east-4.myhuaweicloud.com/omni/omni_infer-a2-arm:release_v0.5.0|[omni_infer-a2-arm:v0.5.0](https://bucket-omni-infer-wuhu.obs.myhuaweicloud.com:443/DockerImage/Omni-Infer/Release/v0.5.0/ARM/omni_infer-a2-arm-v0.5.0.tar?AccessKeyId=HPUABVUGOTP2OPODPEKP\u0026Expires=1789647146\u0026Signature=Ovj1kCPS6OWy/2c4SuVN1rEF0Y4%3D)|\r\n| A2| x86| docker pull swr.cn-east-4.myhuaweicloud.com/omni/omni_infer-a2-x86:release_v0.5.0|[omni_infer-a2-x86:v0.5.0](https://bucket-omni-infer-wuhu.obs.myhuaweicloud.com:443/DockerImage/Omni-Infer/Release/v0.5.0/X86/omni_infer-a2-x86-v0.5.0.tar?AccessKeyId=HPUABVUGOTP2OPODPEKP\u0026Expires=1789647335\u0026Signature=UDdizej5VIj2ubI/mej0e8UA8wc%3D)|\r\n","author":{"name":"liujianxin","username":"octol","path":"/octol","avatar_url":"no_portrait.png#liujianxin-octol"},"attach_files":[],"zip_download_url":"/omniai/omniinfer/releases/tag/v0.5.0.zip","tar_download_url":"/omniai/omniinfer/releases/tag/v0.5.0.tar.gz"}}}