登录 注册
开源 企业版 高校版 私有云 模力方舟 Gitee AI
开源项目 > 人工智能 > AI-人工智能 &&
扫描微信二维码支付
取消
支付完成
Watch
不关注 关注所有动态 仅关注版本发行动态 关注但不提醒动态
10 Star 21 Fork 12

DeepSpark/DeepSparkInference

代码 Issues 5 Pull Requests 0 Wiki 统计 流水线
服务
加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目，私有仓库也完全免费 ：）
免费加入
克隆/下载
分支 18
标签 4
undefined
贡献代码
同步代码
创建 Pull Request
了解更多
对比差异 通过 Pull Request 同步
同步更新到分支
通过 Pull Request 同步
将会在向当前分支创建一个 Pull
Request，合入后将完成同步
取消
提示: 由于 Git 不支持空文件夾，创建文件夹后会生成空的 .keep 文件
.gitee
update .gitee/PULL_REQUEST_TEMPLATE.zh-CN.md.
2个月前
data/datasets
add default datasets loacation at PROJ_ROOT/data/datasests
8个月前
docs
update supported environments of all models
5天前
models
update supported environments of all models
5天前
tests
update clip ci path
22天前
toolbox/ByteMLPerf
fix byte infer
2个月前
.gitignore
update .gitignore to ignore onnx pth and engine
2个月前
CONTRIBUTING.md
initial commit for deepsparkinferce first release
12个月前
LICENSE
initial commit for deepsparkinferce first release
12个月前
README.md
add iluvatar sdk info
12天前
RELEASE.md
fix #IBA0F4change rec_svtr name
3个月前
Loading...
README
Apache-2.0

DeepSparkInference

DeepSparkInference推理模型库作为DeepSpark开源社区的核心项目，于2024年3月正式开源，一期甄选了48个推理模型示例，涵盖计算机视觉，自然语言处理，语音识别等领域，后续将逐步拓展更多AI领域。

DeepSparkInference中的模型提供了在国产推理引擎IGIE或IxRT下运行的推理示例和指导文档，部分模型提供了基于国产通用GPU智铠100的评测结果。

IGIE（Iluvatar GPU Inference Engine）是基于TVM框架研发的高性能、高通用、全流程的AI推理引擎。支持多框架模型导入、量化、图优化、多算子库支持、多后端支持、算子自动调优等特性，为推理场景提供易部署、高吞吐量、低延迟的完整方案。

IxRT（Iluvatar CoreX RunTime）是天数智芯自研的高性能推理引擎，专注于最大限度发挥天数智芯通用GPU 的性能，实现各领域模型的高性能推理。IxRT支持动态形状推理、插件和INT8/FP16推理等特性。

DeepSparkInference将按季度进行版本更新，后续会逐步丰富模型类别并拓展大模型推理。

LLM (Large Language Model)

Model vLLM TRT-LLM TGI IXUCA SDK
Baichuan2-7B ✅ 4.2.0
ChatGLM-3-6B ✅ 4.2.0
ChatGLM-3-6B-32K ✅ 4.2.0
DeepSeek-R1-Distill-Llama-8B ✅ 4.2.0
DeepSeek-R1-Distill-Llama-70B ✅ 4.2.0
DeepSeek-R1-Distill-Qwen-1.5B ✅ 4.2.0
DeepSeek-R1-Distill-Qwen-7B ✅ 4.2.0
DeepSeek-R1-Distill-Qwen-14B ✅ 4.2.0
DeepSeek-R1-Distill-Qwen-32B ✅ 4.2.0
Llama2-7B ✅ ✅ 4.2.0
Llama2-13B ✅ 4.2.0
Llama2-70B ✅ 4.2.0
Llama3-70B ✅ 4.2.0
Qwen-7B ✅ 4.2.0
Qwen1.5-7B ✅ ✅ 4.2.0
Qwen1.5-14B ✅ 4.2.0
Qwen1.5-32B Chat ✅ 4.2.0
Qwen1.5-72B ✅ 4.2.0
Qwen2-7B Instruct ✅ 4.2.0
Qwen2-72B Instruct ✅ 4.2.0
StableLM2-1.6B ✅ 4.2.0

Computer Vision

Classification

Model Prec. IGIE IxRT IXUCA SDK
AlexNet FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
CLIP FP16 ✅ 4.2.0
Conformer-B FP16 ✅ 4.2.0
ConvNeXt-Base FP16 ✅ ✅ 4.2.0
ConvNext-S FP16 ✅ 4.2.0
ConvNeXt-Small FP16 ✅ ✅ 4.2.0
CSPDarkNet53 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
CSPResNet50 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
DeiT-tiny FP16 ✅ ✅ 4.2.0
DenseNet121 FP16 ✅ ✅ 4.2.0
DenseNet161 FP16 ✅ ✅ 4.2.0
DenseNet169 FP16 ✅ ✅ 4.2.0
DenseNet201 FP16 ✅ ✅ 4.2.0
EfficientNet-B0 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
EfficientNet-B1 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
EfficientNet-B2 FP16 ✅ ✅ 4.2.0
EfficientNet-B3 FP16 ✅ ✅ 4.2.0
EfficientNet-B4 FP16 ✅ 4.2.0
EfficientNetV2 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
EfficientNetv2_rw_t FP16 ✅ ✅ 4.2.0
EfficientNetv2_s FP16 ✅ ✅ 4.2.0
GoogLeNet FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
HRNet-W18 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
InceptionV3 FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
Inception-ResNet-V2 FP16 ✅ 4.2.0
INT8 ✅ 4.2.0
Mixer_B FP16 ✅ 4.2.0
MNASNet0_5 FP16 ✅ 4.2.0
MNASNet0_75 FP16 ✅ 4.2.0
MobileNetV2 FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
MobileNetV3_Large FP16 ✅ 4.2.0
MobileNetV3_Small FP16 ✅ ✅ 4.2.0
MViTv2_base FP16 ✅ 4.2.0
RegNet_x_16gf FP16 ✅ 4.2.0
RegNet_x_1_6gf FP16 ✅ 4.2.0
RegNet_y_1_6gf FP16 ✅ 4.2.0
RepVGG FP16 ✅ ✅ 4.2.0
Res2Net50 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
ResNeSt50 FP16 ✅ 4.2.0
ResNet101 FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
ResNet152 FP16 ✅ 4.2.0
INT8 ✅ 4.2.0
ResNet18 FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
ResNet34 FP16 ✅ 4.2.0
INT8 ✅ 4.2.0
ResNet50 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
ResNetV1D50 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
ResNeXt50_32x4d FP16 ✅ ✅ 4.2.0
ResNeXt101_64x4d FP16 ✅ 4.2.0
ResNeXt101_32x8d FP16 ✅ 4.2.0
SEResNet50 FP16 ✅ 4.2.0
ShuffleNetV1 FP16 ✅ 4.2.0
ShuffleNetV2_x0_5 FP16 ✅ 4.2.0
ShuffleNetV2_x1_0 FP16 ✅ 4.2.0
ShuffleNetV2_x1_5 FP16 ✅ 4.2.0
ShuffleNetV2_x2_0 FP16 ✅ 4.2.0
SqueezeNet 1.0 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
SqueezeNet 1.1 FP16 ✅ 4.2.0
INT8 ✅ 4.2.0
SVT Base FP16 ✅ 4.2.0
Swin Transformer FP16 ✅ 4.2.0
Swin Transformer Large FP16 ✅ 4.2.0
VGG11 FP16 ✅ 4.2.0
VGG16 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
Wide ResNet50 FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
Wide ResNet101 FP16 ✅ 4.2.0

Object Detection

Model Prec. IGIE IxRT IXUCA SDK
ATSS FP16 ✅ 4.2.0
CenterNet FP16 ✅ ✅ 4.2.0
DETR FP16 ✅ 4.2.0
FCOS FP16 ✅ ✅ 4.2.0
FoveaBox FP16 ✅ ✅ 4.2.0
FSAF FP16 ✅ ✅ 4.2.0
HRNet FP16 ✅ ✅ 4.2.0
PAA FP16 ✅ 4.2.0
RetinaFace FP16 ✅ ✅ 4.2.0
RetinaNet FP16 ✅ 4.2.0
RTMDet FP16 ✅ 4.2.0
SABL FP16 ✅ 4.2.0
YOLOv3 FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
YOLOv4 FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
YOLOv5 FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
YOLOv5s FP16 ✅ 4.2.0
INT8 ✅ 4.2.0
YOLOv6 FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
YOLOv7 FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
YOLOv8 FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
YOLOv9 FP16 ✅ 4.2.0
YOLOv10 FP16 ✅ 4.2.0
YOLOv11 FP16 ✅ 4.2.0
YOLOX FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0

Face Recognition

Model Prec. IGIE IxRT IXUCA SDK
FaceNet FP16 ✅ 4.2.0
INT8 ✅ 4.2.0

OCR (Optical Character Recognition)

Model Prec. IGIE IXUCA SDK
Kie_layoutXLM FP16 ✅ 4.2.0
SVTR FP16 ✅ 4.2.0

Pose Estimation

Model Prec. IGIE IxRT IXUCA SDK
HRNetPose FP16 ✅ 4.2.0
Lightweight OpenPose FP16 ✅ 4.2.0
RTMPose FP16 ✅ ✅ 4.2.0

Instance Segmentation

Model Prec. IGIE IxRT IXUCA SDK
Mask R-CNN FP16 ✅ 4.2.0
SOLOv1 FP16 ✅ 4.2.0

Multi-Object Tracking

Model Prec. IGIE IxRT IXUCA SDK
FastReID FP16 ✅ 4.2.0
DeepSort FP16 ✅ 4.2.0
INT8 ✅ 4.2.0
RepNet-Vehicle-ReID FP16 ✅ 4.2.0

Multimodal

Model vLLM IxFormer IXUCA SDK
Chameleon-7B ✅ 4.2.0
CLIP ✅ 4.2.0
Fuyu-8B ✅ 4.2.0
InternVL2-4B ✅ 4.2.0
LLaVA ✅ 4.2.0
LLaVA-Next-Video-7B ✅ 4.2.0
MiniCPM V2 ✅ 4.2.0

NLP

PLM (Pre-trained Language Model)

Model Prec. IGIE IxRT IXUCA SDK
ALBERT FP16 ✅ 4.2.0
BERT Base NER INT8 ✅ 4.2.0
BERT Base SQuAD FP16 ✅ ✅ 4.2.0
INT8 ✅ 4.2.0
BERT Large SQuAD FP16 ✅ ✅ 4.2.0
INT8 ✅ ✅ 4.2.0
DeBERTa FP16 ✅ 4.2.0
RoBERTa FP16 ✅ 4.2.0
RoFormer FP16 ✅ 4.2.0
VideoBERT FP16 ✅ 4.2.0

Audio

Speech Recognition

Model Prec. IGIE IxRT IXUCA SDK
Conformer FP16 ✅ ✅ 4.2.0
Transformer ASR FP16 ✅ 4.2.0

Others

Recommendation Systems

Model Prec. IGIE IxRT IXUCA SDK
Wide & Deep FP16 ✅ 4.2.0

社区

治理

请参见 DeepSpark Code of Conduct on Gitee or on GitHub。

交流

请联系 contact@deepspark.org.cn。

贡献

请参见 DeepSparkInference Contributing Guidelines。

免责声明

DeepSparkInference仅提供公共数据集的下载和预处理脚本。这些数据集不属于DeepSparkInference，DeepSparkInference也不对其质量或维护负责。请确保您具有这些数据集的使用许可，基于这些数据集训练的模型仅可用于非商业研究和教育。

致数据集所有者：

如果不希望您的数据集公布在DeepSparkInference上或希望更新DeepSparkInference中属于您的数据集，请在Gitee或Github上提交issue，我们将按您的issue删除或更新。衷心感谢您对我们社区的支持和贡献。

许可证

本项目许可证遵循Apache-2.0。
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Starred
21
Star
21
Fork
12

简介

DeepSparkInference推理模型示例库一期甄选了48个推理模型示例，涵盖计算机视觉，自然语言处理，语音识别等领域，后续将逐步拓展更多AI领域。 展开 收起
暂无标签
Python 等 6 种语言
Python
54.6%
C++
32.9%
Shell
7.6%
Perl
2.7%
CMake
1.0%
Other
1.2%
Apache-2.0
使用 Apache-2.0 开源许可协议
取消

发行版 (4)

全部
DeepSparkInference 24.12 Release
3个月前

DeepSparkInference 开源评估指数

开源评估指数源自 OSS-Compass 评估体系，评估体系围绕以下三个维度对项目展开评估：

1. 开源生态

  • 生产力：来评估开源项目输出软件制品和开源价值的能力。
  • 创新力：用于评估开源软件及其生态系统的多样化程度。
  • 稳健性：用于评估开源项目面对多变的发展环境，抵御内外干扰并自我恢复的能力。

2. 协作、人、软件

  • 协作：代表了开源开发行为中协作的程度和深度。
  • 人：观察开源项目核心人员在开源项目中的影响力，并通过第三方视角考察用户和开发者对开源项目的评价。
  • 软件：从开源项目对外输出的制品评估其价值最终落脚点。也是开源评估最“古老”的主流方向之一“开源软件” 的具体表现。

3. 评估模型

    基于“开源生态”与“协作、人、软件”的维度，找到与该目标直接或间接相关的可量化指标，对开源项目健康与生态进行量化评估，最终形成开源评估指数。
productivity 生产力
niche_creation 创新力
robustness 稳健性
collaboration 协作
contributor 贡献者
software 软件

贡献者 (7)

全部
majorli6 honglyua YoungPeng xinchi.tian may 张文风 郝燕龙

近期动态

majorli6
5天前推送了新的提交到 master 分支，a05d6cd...157e4a5
majorli6
12天前合并了 PR #144 add iluvatar sdk info
majorli6
12天前推送了新的提交到 master 分支，21e5317...a05d6cd
majorli6
12天前创建了 PR #144 add iluvatar sdk info
majorli6
12天前推送了新的提交到 update_sdk 分支，c3799ea...e8f4006
加载更多
不能加载更多了
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
Python
1
https://gitee.com/deep-spark/deepsparkinference.git
git@gitee.com:deep-spark/deepsparkinference.git
deep-spark
deepsparkinference
DeepSparkInference
master
点此查找更多帮助

搜索帮助

Git 命令在线学习 如何在 Gitee 导入 GitHub 仓库
Git 仓库基础操作
企业版和社区版功能对比
SSH 公钥设置
如何处理代码冲突
仓库体积过大，如何减小？
如何找回被删除的仓库数据
Gitee 产品配额说明
GitHub仓库快速导入Gitee及同步更新
什么是 Release（发行版）
将 PHP 项目自动发布到 packagist.org
评论
仓库举报
回到顶部