# yolov8_detect_py

**Repository Path**: mr-hanlu/yolov8_detect_py

## Basic Information

- **Project Name**: yolov8_detect_py
- **Description**: yolov8 检测和分割 onnxruntime openvino tensorrt python推理代码
- **Primary Language**: Python
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 1
- **Created**: 2023-07-17
- **Last Updated**: 2025-02-23

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README


### yolov8分割检测关键点模型推理速度对比

<table>
    <tr>
        <td colspan="7"><center><b>yolov8_s 80类[3*640*640]图推理时间对比_i5-10400F+1650显卡</b><center></td>
    </tr>
    <tr>
        <td width="120px"><center>推理框架</center></td>   <!--合并行-->
        <td width="120px"><center>推理语言</center></td>
        <td width="120px"><center>CPU/GPU</center></td>
        <td><center>detect推理time(ms)</center></td>
        <td><center>segment推理time(ms)</center></td>
        <td><center>keypoint推理time(ms)</center></td>
        <td><center>备注</center></td>
    </tr> 
    <!-- pytorch -->
    <tr>
        <td rowspan="2"><center>Pytorch</center></td>
        <td rowspan="2"><center>python</center></td>
        <td><center>CPU</center></td>
        <td><center>160</center></td>
        <td><center>225</center></td>
        <td><center>170</center></td>
        <td></td>
    </tr>
    <tr>
        <td><center>GPU</center></td>
        <td><center>20</center></td>
        <td><center>26</center></td>
        <td><center>21</center></td>
        <td>检测图大小是动态形状，加最小黑边，所以快</td>
    </tr>
    <!-- onnxruntime -->
    <tr>
        <td rowspan="3"><center>OnnxRuntime</center></td>   <!--合并行-->
        <td rowspan="3"><center>python</center></td>
        <td><center>CPU</center></td>
        <td><center>110</center></td>
        <td><center>150</center></td>
        <td><center>120</center></td>
        <td><center></center></td>
    </tr>
    <tr>
        <td><center>GPU</center></td>
        <td><center>29</center></td>
        <td><center>37</center></td>
        <td><center>30</center></td>
        <td><center>以detect为例，python和c++速度一样</center></td>
    </tr>
    <tr>
        <td><center>Tensorrt</center></td>
        <td><center>24</center></td>
        <td><center>31</center></td>
        <td><center>24</center></td>
        <td>启动速度慢，第一次推理时间很长，GPU占用率高于直接用GPU</td>
    </tr>  
     <!-- openvino -->
    <tr>
        <td rowspan="1"><center>openvino</center></td>   <!--合并行-->
        <td rowspan="1"><center>python</center></td>
        <td><center>CPU</center></td>
        <td><center>65</center></td>
        <td><center>95</center></td>
        <td><center>70</center></td>
        <td><center></center></td>
    </tr>
    <!-- tensorrt -->
    <tr>
        <td rowspan="6"><center>TensorRT</center></td>   <!--合并行-->
        <td rowspan="3"><center>python</center></td>
        <td><center>GPU_int8</center></td>
        <td><center>15</center></td>
        <td><center>19</center></td>
        <td><center>15</center></td>
        <td><center>int8量化完小目标可能会丢</center></td>
    </tr>
    <tr>
        <td><center>GPU_fp16</center></td>
        <td><center>17</center></td>
        <td><center>22</center></td>
        <td><center>17</center></td>
        <td><center></center></td>
    </tr>
    <tr>
        <td><center>GPU_fp32</center></td>
        <td><center>24</center></td>
        <td><center>30</center></td>
        <td><center>24</center></td>
        <td><center></center></td>
    </tr>
    <tr>
        <td rowspan="3"><center>C++</center></td>
        <td><center>GPU_int8</center></td>
        <td><center>13</center></td>
        <td><center>19</center></td>
        <td><center>13</center></td>
        <td><center></center></td>
    </tr>
    <tr>
        <td><center>GPU_fp16</center></td>
        <td><center>15</center></td>
        <td><center>23</center></td>
        <td><center>15</center></td>
        <td><center></center></td>
    </tr>
    <tr>
        <td><center>GPU_fp32</center></td>
        <td><center>22</center></td>
        <td><center>30</center></td>
        <td><center>22</center></td>
        <td><center>c++前处理和网络推理比python各快1ms</center></td>
    </tr>
</table>



**总结：**
1. python部署图片固定大小用OnnxRuntime(量化fp16速度变慢，量化int8精度丢失严重，直接使用默认模型)
2. 服务端CPU部署使用OpenVINO
3. 服务端GPU部署使用TensorRT（FP16速度快，int8会损失部分小目标，需要针对任务实际测试）