@@ -259,28 +282,132 @@
- 测试数据均使用高通骁龙865(4\*A77 + 4\*A55)处理器batch size为1, 开启4线程测试，测试使用NCNN预测库，测试脚本见[MobileDetBenchmark](https://github.com/JiweiMaster/MobileDetBenchmark)
- [PP-PicoDet](configs/picodet)及[PP-YOLO-Tiny](configs/ppyolo)为PaddleDetection自研模型，其余模型PaddleDetection暂未提供
-## 文档教程
+
+
+##
模型库
+
+
+ 1. 通用检测
+
+#### [PP-YOLOE](./configs/ppyoloe)系列 推荐场景：Nvidia V100, T4等云端GPU和Jetson系列等边缘端设备
+
+| 模型名称 | COCO精度（mAP） | V100 TensorRT FP16速度(FPS) | 配置文件 | 模型下载 |
+|:---------- |:-----------:|:-------------------------:|:-----------------------------------------------------:|:------------------------------------------------------------------------------------:|
+| PP-YOLOE-s | 42.7 | 333.3 | [链接](configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams) |
+| PP-YOLOE-m | 48.6 | 208.3 | [链接](configs/ppyolo/ppyolo_r50vd_dcn_2x_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/ppyolo_r50vd_dcn_2x_coco.pdparams) |
+| PP-YOLOE-l | 50.9 | 149.2 | [链接](configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams) |
+| PP-YOLOE-x | 51.9 | 95.2 | [链接](configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams) |
+
+#### [PP-PicoDet](./configs/picodet)系列 推荐场景：ARM CPU(RK3399, 树莓派等) 和NPU(比特大陆，晶晨等)移动端芯片和x86 CPU设备
+
+| 模型名称 | COCO精度（mAP） | 骁龙865 四线程速度(ms) | 配置文件 | 模型下载 |
+|:---------- |:-----------:|:---------------:|:---------------------------------------------------:|:---------------------------------------------------------------------------------:|
+| PicoDet-XS | 23.5 | 7.81 | [链接](configs/picodet/picodet_xs_320_coco_lcnet.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/picodet_xs_320_coco_lcnet.pdparams) |
+| PicoDet-S | 29.1 | 9.56 | [链接](configs/picodet/picodet_s_320_coco_lcnet.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/picodet_s_320_coco_lcnet.pdparams) |
+| PicoDet-M | 34.4 | 17.68 | [链接](configs/picodet/picodet_m_320_coco_lcnet.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/picodet_m_320_coco_lcnet.pdparams) |
+| PicoDet-L | 36.1 | 25.21 | [链接](configs/picodet/picodet_l_320_coco_lcnet.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/picodet_l_320_coco_lcnet.pdparams) |
+
+#### 前沿检测算法
+
+| 模型名称 | COCO精度（mAP） | V100 TensorRT FP16速度(FPS) | 配置文件 | 模型下载 |
+|:------------------------------------------------------------------ |:-----------:|:-------------------------:|:------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------:|
+| [YOLOX-l](configs/yolox) | 50.1 | 107.5 | [链接](configs/yolox/yolox_l_300e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/yolox_l_300e_coco.pdparams) |
+| [YOLOv5-l](https://github.com/nemonameless/PaddleDetection_YOLOSeries/tree/develop/configs/yolov5) | 48.6 | 136.0 | [链接](https://github.com/nemonameless/PaddleDetection_YOLOSeries/blob/develop/configs/yolov5/yolov5_l_300e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/yolov5_l_300e_coco.pdparams) |
+| [YOLOv7-l](https://github.com/nemonameless/PaddleDetection_YOLOSeries/tree/develop/configs/yolov7) | 51.0 | 135.0 | [链接](https://github.com/nemonameless/PaddleDetection_YOLOSeries/blob/develop/configs/yolov7/yolov7_l_300e_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/yolov7_l_300e_coco.pdparams) |
+
+#### 其他通用检测模型 [文档链接](docs/MODEL_ZOO_cn.md)
+
+
+
+
+ 2. 实例分割
+
+| 模型名称 | 模型简介 | 推荐场景 | COCO精度(mAP) | 配置文件 | 模型下载 |
+|:----------------- |:------------ |:---- |:--------------------------------:|:---------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------:|
+| Mask RCNN | 两阶段实例分割算法 | 云边端 | box AP: 41.4
mask AP: 37.5 | [链接](configs/mask_rcnn/mask_rcnn_r50_vd_fpn_2x_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/mask_rcnn_r50_vd_fpn_2x_coco.pdparams) |
+| Cascade Mask RCNN | 两阶段实例分割算法 | 云边端 | box AP: 45.7
mask AP: 39.7 | [链接](configs/mask_rcnn/cascade_mask_rcnn_r50_vd_fpn_ssld_2x_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/cascade_mask_rcnn_r50_vd_fpn_ssld_2x_coco.pdparams) |
+| SOLOv2 | 轻量级单阶段实例分割算法 | 云边端 | mask AP: 38.0 | [链接](configs/solov2/solov2_r50_fpn_3x_coco.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/solov2_r50_fpn_3x_coco.pdparams) |
+
+
+
+
+ 3. 关键点检测
+
+| 模型名称 | 模型简介 | 推荐场景 | COCO精度（AP） | 速度 | 配置文件 | 模型下载 |
+|:------------------------------------------- |:---------------------------------------------------------------- |:---------------------------------- |:----------:|:-----------------------:|:-------------------------------------------------------:|:---------------------------------------------------------------------------------------:|
+| HRNet-w32 + DarkPose | top-down 关键点检测算法
输入尺寸384x288
| 云边端
| 78.3 | T4 TensorRT FP16 2.96ms | [链接](configs/keypoint/hrnet/dark_hrnet_w32_384x288.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/keypoint/dark_hrnet_w32_384x288.pdparams) |
+| HRNet-w32 + DarkPose | top-down 关键点检测算法
输入尺寸256x192 | 云边端 | 78.0 | T4 TensorRT FP16 1.75ms | [链接](configs/keypoint/hrnet/dark_hrnet_w32_256x192.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/keypoint/dark_hrnet_w32_256x192.pdparams) |
+| [PP-TinyPose](./configs/keypoint/tiny_pose) | 轻量级关键点算法
输入尺寸256x192 | 移动端 | 68.8 | 骁龙865 四线程 6.30ms | [链接](configs/keypoint/tiny_pose/tinypose_256x192.yml) | [下载地址](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192.pdparams) |
+| [PP-TinyPose](./configs/keypoint/tiny_pose) | 轻量级关键点算法
输入尺寸128x96 | 移动端 | 58.1 | 骁龙865 四线程 2.37ms | [链接](configs/keypoint/tiny_pose/tinypose_128x96.yml) | [下载地址](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96.pdparams) |
+
+#### 其他关键点检测模型 [文档链接](configs/keypoint)
+
+
+
+
+ 4. 多目标跟踪PP-Tracking
+
+| 模型名称 | 模型简介 | 推荐场景 | 精度 | 配置文件 | 模型下载 |
+|:--------- |:------------------------ |:---------------------------------- |:----------------------:|:---------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------:|
+| DeepSORT | SDE多目标跟踪算法 检测、ReID模型相互独立 | 云边端
| MOT-17 half val: 66.9 | [链接](configs/mot/deepsort/deepsort_jde_yolov3_pcb_pyramid.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/mot/deepsort/deepsort_pcb_pyramid_r101.pdparams) |
+| ByteTrack | SDE多目标跟踪算法 仅包含检测模型 | 云边端 | MOT-17 half val: 77.3 | [链接](configs/mot/bytetrack/detector/yolox_x_24e_800x1440_mix_det.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/mot/deepsort/yolox_x_24e_800x1440_mix_det.pdparams) |
+| JDE | JDE多目标跟踪算法 多任务联合学习方法 | 云边端 | MOT-16 test: 64.6 | [链接](configs/mot/jde/jde_darknet53_30e_1088x608.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/mot/jde_darknet53_30e_1088x608.pdparams) |
+| FairMOT | JDE多目标跟踪算法 多任务联合学习方法 | 云边端 | MOT-16 test: 75.0 | [链接](configs/mot/fairmot/fairmot_dla34_30e_1088x608.yml) | [下载地址](https://paddledet.bj.bcebos.com/models/mot/fairmot_dla34_30e_1088x608.pdparams) |
+
+#### 其他多目标跟踪模型 [文档链接](configs/mot)
+
+
+
+
+ 5. 产业级实时行人分析工具
+
+
+| 任务 | 端到端速度（ms）| 模型方案 | 模型体积 |
+| :---------: | :-------: | :------: |:------: |
+| 行人检测（高精度） | 25.1ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M |
+| 行人检测（轻量级） | 16.2ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
+| 行人跟踪（高精度） | 31.8ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M |
+| 行人跟踪（轻量级） | 21.0ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
+| 属性识别（高精度） | 单人8.5ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPHGNet_small_person_attribute_954_infer.zip) | 目标检测：182M
属性识别：86M |
+| 属性识别（轻量级） | 单人7.1ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip) | 目标检测：182M
属性识别：86M |
+| 摔倒识别 | 单人10ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[关键点检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip)
[基于关键点行为识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | 多目标跟踪：182M
关键点检测：101M
基于关键点行为识别：21.8M |
+| 闯入识别 | 31.8ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M |
+| 打架识别 | 19.7ms | [视频分类](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 90M |
+| 抽烟识别 | 单人15.1ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[基于人体id的目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/ppyoloe_crn_s_80e_smoking_visdrone.zip) | 目标检测：182M
基于人体id的目标检测：27M |
+| 打电话识别 | 单人ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[基于人体id的图像分类](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPHGNet_tiny_calling_halfbody.zip) | 目标检测：182M
基于人体id的图像分类：45M |
+
+
+点击模型方案中的模型即可下载指定模型
+
+详细信息参考[文档](deploy/pipeline)
+
+
+
+
+##
文档教程
### 入门教程
- [安装说明](docs/tutorials/INSTALL_cn.md)
-- [数据准备](docs/tutorials/PrepareDataSet.md)
-- [30分钟上手PaddleDetecion](docs/tutorials/GETTING_STARTED_cn.md)
+- [快速体验](docs/tutorials/QUICK_STARTED_cn.md)
+- [数据准备](docs/tutorials/data/README.md)
+- [PaddleDetection全流程使用](docs/tutorials/GETTING_STARTED_cn.md)
+- [自定义数据训练](docs/tutorials/CustomizeDataTraining.md)
- [FAQ/常见问题汇总](docs/tutorials/FAQ)
### 进阶教程
- 参数配置
-
+
- [RCNN参数说明](docs/tutorials/config_annotation/faster_rcnn_r50_fpn_1x_coco_annotation.md)
- [PP-YOLO参数说明](docs/tutorials/config_annotation/ppyolo_r50vd_dcn_1x_coco_annotation.md)
- 模型压缩(基于[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim))
-
+
- [剪裁/量化/蒸馏教程](configs/slim)
- [推理部署](deploy/README.md)
-
+
- [模型导出教程](deploy/EXPORT_MODEL.md)
- [Paddle Inference部署](deploy/README.md)
- [Python端推理部署](deploy/python)
@@ -291,51 +418,44 @@
- [推理benchmark](deploy/BENCHMARK_INFER.md)
- 进阶开发
-
+
- [数据处理模块](docs/advanced_tutorials/READER.md)
- [新增检测模型](docs/advanced_tutorials/MODEL_TECHNICAL.md)
+ - 二次开发教程
+ - [目标检测](docs/advanced_tutorials/customization/detection.md)
+ - [关键点检测](docs/advanced_tutorials/customization/keypoint_detection.md)
+ - [多目标跟踪](docs/advanced_tutorials/customization/pphuman_mot.md)
+ - [行为识别](docs/advanced_tutorials/customization/pphuman_action.md)
+ - [属性识别](docs/advanced_tutorials/customization/pphuman_attribute.md)
+
+### 课程专栏
+
+- **【理论基础】[目标检测7日打卡营](https://aistudio.baidu.com/aistudio/education/group/info/1617)：** 目标检测任务综述、RCNN系列目标检测算法详解、YOLO系列目标检测算法详解、PP-YOLO优化策略与案例分享、AnchorFree系列算法介绍和实践
+
+- **【产业实践】[AI快车道产业级目标检测技术与应用](https://aistudio.baidu.com/aistudio/education/group/info/23670)：** 目标检测超强目标检测算法矩阵、实时行人分析系统PP-Human、目标检测产业应用全流程拆解与实践
+
+- **【行业特色】2022.3.26 [智慧城市行业七日课](https://aistudio.baidu.com/aistudio/education/group/info/25620)：** 城市规划、城市治理、智慧政务、交通管理、社区治理
+
+### [产业实践范例教程](./industrial_tutorial/README.md)
+
+- [基于PP-TinyPose增强版的智能健身动作识别](https://aistudio.baidu.com/aistudio/projectdetail/4385813)
+
+- [基于PP-Human的打架识别](https://aistudio.baidu.com/aistudio/projectdetail/4086987?contributionType=1)
+
+- [基于PP-PicoDet增强版的路面垃圾检测](https://aistudio.baidu.com/aistudio/projectdetail/3846170?channelType=0&channel=0)
+
+- [基于PP-PicoDet的通信塔识别及Android端部署](https://aistudio.baidu.com/aistudio/projectdetail/3561097)
+
+- [基于FairMOT实现人流量统计](https://aistudio.baidu.com/aistudio/projectdetail/2421822)
+
+- [更多其他范例](./industrial_tutorial/README.md)
+
+##
应用案例
-## 模型库
-
-- 通用目标检测:
- - [模型库](docs/MODEL_ZOO_cn.md)
- - [PP-YOLOE模型](configs/ppyoloe/README_cn.md)
- - [PP-YOLO模型](configs/ppyolo/README_cn.md)
- - [PP-PicoDet模型](configs/picodet/README.md)
- - [增强版Anchor Free模型TTFNet](configs/ttfnet/README.md)
- - [移动端模型](static/configs/mobile/README.md)
- - [676类目标检测](static/docs/featured_model/LARGE_SCALE_DET_MODEL.md)
- - [两阶段实用模型PSS-Det](configs/rcnn_enhance/README.md)
- - [半监督知识蒸馏预训练检测模型](docs/feature_models/SSLD_PRETRAINED_MODEL.md)
-- 通用实例分割
- - [SOLOv2](configs/solov2/README.md)
-- 旋转框检测
- - [S2ANet](configs/dota/README.md)
-- [关键点检测](configs/keypoint)
- - [PP-TinyPose](configs/keypoint/tiny_pose)
- - HigherHRNet
- - HRNet
- - LiteHRNet
-- [多目标跟踪](configs/mot/README.md)
- - [PP-Tracking](deploy/pptracking/README.md)
- - [DeepSORT](configs/mot/deepsort/README_cn.md)
- - [JDE](configs/mot/jde/README_cn.md)
- - [FairMOT](configs/mot/fairmot/README_cn.md)
-- 垂类领域
- - [行人检测](configs/pedestrian/README.md)
- - [车辆检测](configs/vehicle/README.md)
- - [人脸检测](configs/face_detection/README.md)
- - [实时行人分析](deploy/pphuman/README.md)
-- 比赛冠军方案
- - [Objects365 2019 Challenge夺冠模型](static/docs/featured_model/champion_model/CACascadeRCNN.md)
- - [Open Images 2019-Object Detction比赛最佳单模型](static/docs/featured_model/champion_model/OIDV5_BASELINE_MODEL.md)
-
-## 应用案例
-
-- [人像圣诞特效自动生成工具](static/application/christmas)
- [安卓健身APP](https://github.com/zhiboniu/pose_demo_android)
+- [多目标跟踪系统GUI可视化界面](https://github.com/yangyudong2020/PP-Tracking_GUi)
-## 第三方教程推荐
+##
第三方教程推荐
- [PaddleDetection在Windows下的部署(一)](https://zhuanlan.zhihu.com/p/268657833)
- [PaddleDetection在Windows下的部署(二)](https://zhuanlan.zhihu.com/p/280206376)
@@ -343,15 +463,15 @@
- [安全帽检测YOLOv3模型在树莓派上的部署](https://github.com/PaddleCV-FAQ/PaddleDetection-FAQ/blob/main/Lite%E9%83%A8%E7%BD%B2/yolov3_for_raspi.md)
- [使用SSD-MobileNetv1完成一个项目--准备数据集到完成树莓派部署](https://github.com/PaddleCV-FAQ/PaddleDetection-FAQ/blob/main/Lite%E9%83%A8%E7%BD%B2/ssd_mobilenet_v1_for_raspi.md)
-## 版本更新
+##
版本更新
版本更新内容请参考[版本更新文档](docs/CHANGELOG.md)
-## 许可证书
+##
许可证书
本项目的发布受[Apache 2.0 license](LICENSE)许可认证。
-## 贡献代码
+##
贡献代码
我们非常欢迎你可以为PaddleDetection提供代码，也十分感谢你的反馈。
@@ -359,9 +479,10 @@
- 感谢[FL77N](https://github.com/FL77N/)贡献`Sparse-RCNN`模型。
- 感谢[Chen-Song](https://github.com/Chen-Song)贡献`Swin Faster-RCNN`模型。
- 感谢[yangyudong](https://github.com/yangyudong2020), [hchhtc123](https://github.com/hchhtc123) 开发PP-Tracking GUI界面
-- 感谢[Shigure19](https://github.com/Shigure19) 开发PP-TinyPose健身APP
+- 感谢Shigure19 开发PP-TinyPose健身APP
+- 感谢[manangoel99](https://github.com/manangoel99)贡献Wandb可视化方式
-## 引用
+##
引用
```
@misc{ppdet2019,
diff --git a/README_en.md b/README_en.md
index ad02eeeaff1bd594640681c2452e2979c65bc1bf..9fc92960dc4c3b336d12a0a5bce682dd2e101300 100644
--- a/README_en.md
+++ b/README_en.md
@@ -1,39 +1,85 @@
-English | [简体中文](README_cn.md)
+[简体中文](README_cn.md) | English
+
+
+ 
+
+
+**A High-Efficient Development Toolkit for Object Detection based on [PaddlePaddle](https://github.com/paddlepaddle/paddle)**
+
+
+ 
+ 
+ 
+ 
+ 
+
+
+
+
+
+
+
+
+##
Product Update
-# Product news
+- 🔥 **2022.8.09：Release [YOLO series model zoo](https://github.com/nemonameless/PaddleDetection_YOLOSeries)**
+ - Comprehensive coverage of classic and latest models of the YOLO series: Including YOLOv3，Paddle real-time object detection model PP-YOLOE, and frontier detection algorithms YOLOv4, YOLOv5, YOLOX, MT-YOLOv6 and YOLOv7
+ - Better model performance：Upgrade based on various YOLO algorithms, shorten training time in 5-8 times and the accuracy is generally improved by 1%-5% mAP. The model compression strategy is used to achieve 30% improvement in speed without precision loss
+ - Complete end-to-end development support：End-to-end development pipieline including training, evaluation, inference, model compression and deployment on various hardware. Meanwhile, support flexible algorithnm switch and implement customized development efficiently
-- 2021.11.03: Release [release/2.3](https://github.com/PaddlePaddle/Paddleetection/tree/release/2.3) version. Release mobile object detection model ⚡[PP-PicoDet](configs/picodet), mobile keypoint detection model ⚡[PP-TinyPose](configs/keypoint/tiny_pose)，Real-time tracking system [PP-Tracking](deploy/pptracking). Release object detection models, including [Swin-Transformer](configs/faster_rcnn), [TOOD](configs/tood), [GFL](configs/gfl), release [Sniper](configs/sniper) tiny object detection models and optimized [PP-YOLO-EB](configs/ppyolo) model for EdgeBoard. Release mobile keypoint detection model [Lite HRNet](configs/keypoint).
-- 2021.08.10: Release [release/2.2](https://github.com/PaddlePaddle/Paddleetection/tree/release/2.2) version. Release Transformer object detection models, including [DETR](configs/detr), [Deformable DETR](configs/deformable_detr), [Sparse RCNN](configs/sparse_rcnn). Release [keypoint detection](configs/keypoint) models, including DarkHRNet and model trained on MPII dataset. Release [head-tracking](configs/mot/headtracking21) and [vehicle-tracking](configs/mot/vehicle) multi-object tracking models.
-- 2021.05.20: Release [release/2.1](https://github.com/PaddlePaddle/Paddleetection/tree/release/2.1) version. Release [Keypoint Detection](configs/keypoint), including HigherHRNet and HRNet, [Multi-Object Tracking](configs/mot), including DeepSORT，JDE and FairMOT. Release model compression for PPYOLO series models.Update documents such as [EXPORT ONNX MODEL](deploy/EXPORT_ONNX_MODEL.md).
+- 🔥 **2022.8.01：Release [PP-TinyPose plus](./configs/keypoint/tiny_pose/). The end-to-end precision improves 9.1% AP in dataset
+ of fitness and dance scenes**
+ - Increase data of sports scenes, and the recognition performance of complex actions is significantly improved, covering actions such as sideways, lying down, jumping, and raising legs
+ - Detection model uses PP-PicoDet plus and the precision on COCO dataset is improved by 3.1% mAP
+ - The stability of keypoints is enhanced. Implement the filter stabilization method to make the video prediction result more stable and smooth.
+- 2022.7.14：Release [pedestrian analysis tool PP-Human v2](./deploy/pipeline)
+ - Four major functions: five complicated action recognition with high performance and Flexible, real-time human attribute recognition, visitor flow statistics and high-accuracy multi-camera tracking.
+ - High performance algorithm: including pedestrian detection, tracking, attribute recognition which is robust to the number of targets and the variant of background and light.
+ - Highly Flexible: providing complete introduction of end-to-end development and optimization strategy, simple command for deployment and compatibility with different input format.
-# Introduction
+- 2022.3.24：PaddleDetection released[release/2.4 version](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.4)
+ - Release high-performanace SOTA object detection model [PP-YOLOE](configs/ppyoloe). It integrates cloud and edge devices and provides S/M/L/X versions. In particular, Verson L has the accuracy as 51.4% on COCO test 2017 dataset, inference speed as 78.1 FPS on a single Test V100. It supports mixed precision training, 33% faster than PP-YOLOv2. Its full range of multi-sized models can meet different hardware arithmetic requirements, and adaptable to server, edge-device GPU and other AI accelerator cards on servers.
+ - Release ultra-lightweight SOTA object detection model [PP-PicoDet Plus](configs/picodet) with 2% improvement in accuracy and 63% improvement in CPU inference speed. Add PicoDet-XS model with a 0.7M parameter, providing model sparsification and quantization functions for model acceleration. No specific post processing module is required for all the hardware, simplifying the deployment.
+ - Release the real-time pedestrian analysis tool [PP-Human](deploy/pphuman). It has four major functions: pedestrian tracking, visitor flow statistics, human attribute recognition and falling detection. For falling detection, it is optimized based on real-life data with accurate recognition of various types of falling posture. It can adapt to different environmental background, light and camera angle.
+ - Add [YOLOX](configs/yolox) object detection model with nano/tiny/S/M/L/X. X version has the accuracy as 51.8% on COCO Val2017 dataset.
-PaddleDetection is an end-to-end object detection development kit based on PaddlePaddle, which implements varied mainstream object detection, instance segmentation, tracking and keypoint detection algorithms in modular designwhich with configurable modules such as network components, data augmentations and losses, and release many kinds SOTA industry practice models, integrates abilities of model compression and cross-platform high-performance deployment, aims to help developers in the whole end-to-end development in a faster and better way.
+- [More releases](https://github.com/PaddlePaddle/PaddleDetection/releases)
-### PaddleDetection provides image processing capabilities such as object detection, instance segmentation, multi-object tracking, keypoint detection and etc.
+##
Brief Introduction
-
-
+**PaddleDetection** is an end-to-end object detection development kit based on PaddlePaddle. Providing **over 30 model algorithm** and **over 250 pre-trained models**, it covers object detection, instance segmentation, keypoint detection, multi-object tracking. In particular, PaddleDetection offers **high- performance & light-weight** industrial SOTA models on **servers and mobile** devices, champion solution and cutting-edge algorithm. PaddleDetection provides various data augmentation methods, configurable network components, loss functions and other advanced optimization & deployment schemes. In addition to running through the whole process of data processing, model development, training, compression and deployment, PaddlePaddle also provides rich cases and tutorials to accelerate the industrial application of algorithm.
+
+
+
-### Features
-- **Rich Models**
-PaddleDetection provides rich of models, including **100+ pre-trained models** such as **object detection**, **instance segmentation**, **face detection** etc. It covers a variety of **global competition champion** schemes.
+##
Features
+
+- **Rich model library**: PaddleDetection provides over 250 pre-trained models including **object detection, instance segmentation, face recognition, multi-object tracking**. It covers a variety of **global competition champion** schemes.
+- **Simple to use**: Modular design, decoupling each network component, easy for developers to build and try various detection models and optimization strategies, quick access to high-performance, customized algorithm.
+- **Getting Through End to End**: PaddlePaddle gets through end to end from data augmentation, constructing models, training, compression, depolyment. It also supports multi-architecture, multi-device deployment for **cloud and edge** device.
+- **High Performance**: Due to the high performance core, PaddlePaddle has clear advantages in training speed and memory occupation. It also supports FP16 training and multi-machine training.
+
+
+
+
Exchanges
-- **Production Ready:**
-From data augmentation, constructing models, training, compression, depolyment, get through end to end, and complete support for multi-architecture, multi-device deployment for **cloud and edge device**.
+- If you have any question or suggestion, please give us your valuable input via [GitHub Issues](https://github.com/PaddlePaddle/PaddleDetection/issues)
-- **High Performance:**
-Based on the high performance core of PaddlePaddle, advantages of training speed and memory occupation are obvious. FP16 training and multi-machine training are supported as well.
+ Welcome to join PaddleDetection user groups on QQ, WeChat (scan the QR code, add and reply "D" to the assistant)
+
+
+
+
+
-#### Overview of Kit Structures
+##
Kit Structure
-#### Overview of Model Performance
+##
Model Performance
+
+
+ Performance comparison of Cloud models
-The relationship between COCO mAP and FPS on Tesla V100 of representative models of each server side architectures and backbones.
+The comparison between COCO mAP and FPS on Tesla V100 of representative models of each architectures and backbones.
-
-
- **NOTE:**
-
- - `CBResNet stands` for `Cascade-Faster-RCNN-CBResNet200vd-FPN`, which has highest mAP on COCO as 53.3%
+
- - `Cascade-Faster-RCNN` stands for `Cascade-Faster-RCNN-ResNet50vd-DCN`, which has been optimized to 20 FPS inference speed when COCO mAP as 47.8% in PaddleDetection models
+**Clarification：**
- - `PP-YOLO` achieves mAP of 45.9% on COCO and 72.9FPS on Tesla V100. Both precision and speed surpass [YOLOv4](https://arxiv.org/abs/2004.10934)
+- `CBResNet` stands for `Cascade-Faster-RCNN-CBResNet200vd-FPN`, which has highest mAP on COCO as 53.3%
+- `Cascade-Faster-RCNN`stands for `Cascade-Faster-RCNN-ResNet50vd-DCN`, which has been optimized to 20 FPS inference speed when COCO mAP as 47.8% in PaddleDetection models
+- `PP-YOLO` reached accuracy as 45.9% on COCO dataset, inference speed as 72.9 FPS on Tesla V100, higher than [YOLOv4]([[2004.10934] YOLOv4: Optimal Speed and Accuracy of Object Detection](https://arxiv.org/abs/2004.10934)) in terms of speed and accuracy
+- `PP-YOLO v2`are optimized `PP-YOLO`. It reached accuracy as 49.5% on COCO dataset, inference speed as 68.9 FPS on Tesla V100.
+- `PP-YOLOE`are optimized `PP-YOLO v2`. It reached accuracy as 51.4% on COCO dataset, inference speed as 78.1 FPS on Tesla V100
+- The models in the figure are available in the[ model library](#模型库)
- - `PP-YOLO v2` is optimized version of `PP-YOLO` which has mAP of 49.5% and 68.9FPS on Tesla V100
+
- - All these models can be get in [Model Zoo](#ModelZoo)
+
+ Performance omparison on mobiles
-The relationship between COCO mAP and FPS on Qualcomm Snapdragon 865 of representative mobile side models.
+The comparison between COCO mAP and FPS on Qualcomm Snapdragon 865 processor of models on mobile devices.
-
+
+
+##
Model libraries
+
+
+ 1. General detection
+
+#### PP-YOLOE series Recommended scenarios: Cloud GPU such as Nvidia V100, T4 and edge devices such as Jetson series
+
+| Model | COCO Accuracy（mAP） | V100 TensorRT FP16 Speed(FPS) | Configuration | Download |
+|:---------- |:------------------:|:-----------------------------:|:-------------------------------------------------------:|:----------------------------------------------------------------------------------------:|
+| PP-YOLOE-s | 42.7 | 333.3 | [Link](configs/ppyoloe/ppyoloe_crn_l_300e_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/ppyoloe_crn_l_300e_coco.pdparams) |
+| PP-YOLOE-m | 48.6 | 208.3 | [Link](configs/ppyolo/ppyolo_r50vd_dcn_2x_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/ppyolo_r50vd_dcn_2x_coco.pdparams) |
+| PP-YOLOE-l | 50.9 | 149.2 | [Link](configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams) |
+| PP-YOLOE-x | 51.9 | 95.2 | [Link](configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams) |
+
+#### PP-PicoDet series Recommended scenarios: Mobile chips and x86 CPU devices, such as ARM CPU(RK3399, Raspberry Pi) and NPU(BITMAIN)
+
+| Model | COCO Accuracy（mAP） | Snapdragon 865 four-thread speed (ms) | Configuration | Download |
+|:---------- |:------------------:|:-------------------------------------:|:-----------------------------------------------------:|:-------------------------------------------------------------------------------------:|
+| PicoDet-XS | 23.5 | 7.81 | [Link](configs/picodet/picodet_xs_320_coco_lcnet.yml) | [Download](https://paddledet.bj.bcebos.com/models/picodet_xs_320_coco_lcnet.pdparams) |
+| PicoDet-S | 29.1 | 9.56 | [Link](configs/picodet/picodet_s_320_coco_lcnet.yml) | [Download](https://paddledet.bj.bcebos.com/models/picodet_s_320_coco_lcnet.pdparams) |
+| PicoDet-M | 34.4 | 17.68 | [Link](configs/picodet/picodet_m_320_coco_lcnet.yml) | [Download](https://paddledet.bj.bcebos.com/models/picodet_m_320_coco_lcnet.pdparams) |
+| PicoDet-L | 36.1 | 25.21 | [Link](configs/picodet/picodet_l_320_coco_lcnet.yml) | [Download](https://paddledet.bj.bcebos.com/models/picodet_l_320_coco_lcnet.pdparams) |
+
+#### Frontier detection algorithm
+
+| Model | COCO Accuracy（mAP） | V100 TensorRT FP16 speed(FPS) | Configuration | Download |
+|:-------- |:------------------:|:-----------------------------:|:--------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------:|
+| YOLOX-l | 50.1 | 107.5 | [Link](configs/yolox/yolox_l_300e_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/yolox_l_300e_coco.pdparams) |
+| YOLOv5-l | 48.6 | 136.0 | [Link](https://github.com/nemonameless/PaddleDetection_YOLOv5/blob/main/configs/yolov5/yolov5_l_300e_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/yolov5_l_300e_coco.pdparams) |
+
+#### Other general purpose models [doc](docs/MODEL_ZOO_cn.md)
+
+
+
+
+ 2. Instance segmentation
+
+| Model | Introduction | Recommended Scenarios | COCO Accuracy(mAP) | Configuration | Download |
+|:----------------- |:-------------------------------------------------------- |:--------------------------------------------- |:--------------------------------:|:-----------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------:|
+| Mask RCNN | Two-stage instance segmentation algorithm | Edge-Cloud end
| box AP: 41.4
mask AP: 37.5 | [Link](configs/mask_rcnn/mask_rcnn_r50_vd_fpn_2x_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/mask_rcnn_r50_vd_fpn_2x_coco.pdparams) |
+| Cascade Mask RCNN | Two-stage instance segmentation algorithm | Edge-Cloud end
| box AP: 45.7
mask AP: 39.7 | [Link](configs/mask_rcnn/cascade_mask_rcnn_r50_vd_fpn_ssld_2x_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/cascade_mask_rcnn_r50_vd_fpn_ssld_2x_coco.pdparams) |
+| SOLOv2 | Lightweight single-stage instance segmentation algorithm | Edge-Cloud end
| mask AP: 38.0 | [Link](configs/solov2/solov2_r50_fpn_3x_coco.yml) | [Download](https://paddledet.bj.bcebos.com/models/solov2_r50_fpn_3x_coco.pdparams) |
+
+
+
+
+ 3. Keypoint detection
-- All data tested on Qualcomm Snapdragon 865(4\*A77 + 4\*A55) processor with batch size of 1 and CPU threads of 4, and use NCNN library in testing, benchmark scripts is publiced at [MobileDetBenchmark](https://github.com/JiweiMaster/MobileDetBenchmark)
-- [PP-PicoDet](configs/picodet) and [PP-YOLO-Tiny](configs/ppyolo) are developed and released by PaddleDetection, other models are not provided in PaddleDetection.
+| Model | Introduction | Recommended scenarios | COCO Accuracy（AP） | Speed | Configuration | Download |
+|:-------------------- |:--------------------------------------------------------------------------------------------- |:--------------------------------------------- |:-----------------:|:---------------------------------:|:---------------------------------------------------------:|:-------------------------------------------------------------------------------------------:|
+| HRNet-w32 + DarkPose | Top-down Keypoint detection algorithm
Input size: 384x288
| Edge-Cloud end
| 78.3 | T4 TensorRT FP16 2.96ms | [Link](configs/keypoint/hrnet/dark_hrnet_w32_384x288.yml) | [Download](https://paddledet.bj.bcebos.com/models/keypoint/dark_hrnet_w32_384x288.pdparams) |
+| HRNet-w32 + DarkPose | Top-down Keypoint detection algorithm
Input size: 256x192 | Edge-Cloud end | 78.0 | T4 TensorRT FP16 1.75ms | [Link](configs/keypoint/hrnet/dark_hrnet_w32_256x192.yml) | [Download](https://paddledet.bj.bcebos.com/models/keypoint/dark_hrnet_w32_256x192.pdparams) |
+| PP-TinyPose | Light-weight keypoint algorithm
Input size: 256x192 | Mobile | 68.8 | Snapdragon 865 four-thread 6.30ms | [Link](configs/keypoint/tiny_pose/tinypose_256x192.yml) | [Download](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192.pdparams) |
+| PP-TinyPose | Light-weight keypoint algorithm
Input size: 128x96 | Mobile | 58.1 | Snapdragon 865 four-thread 2.37ms | [Link](configs/keypoint/tiny_pose/tinypose_128x96.yml) | [Download](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96.pdparams) |
-## Tutorials
+#### Other keypoint detection models [doc](configs/keypoint)
-### Get Started
+
-- [Installation guide](docs/tutorials/INSTALL.md)
-- [Prepare dataset](docs/tutorials/PrepareDataSet_en.md)
-- [Quick start on PaddleDetection](docs/tutorials/GETTING_STARTED.md)
+
+ 4. Multi-object tracking PP-Tracking
+| Model | Introduction | Recommended scenarios | Accuracy | Configuration | Download |
+|:--------- |:------------------------------------------------------------- |:--------------------- |:----------------------:|:-----------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------:|
+| DeepSORT | SDE Multi-object tracking algorithm, independent ReID models | Edge-Cloud end | MOT-17 half val: 66.9 | [Link](configs/mot/deepsort/deepsort_jde_yolov3_pcb_pyramid.yml) | [Download](https://paddledet.bj.bcebos.com/models/mot/deepsort/deepsort_pcb_pyramid_r101.pdparams) |
+| ByteTrack | SDE Multi-object tracking algorithm with detection model only | Edge-Cloud end | MOT-17 half val: 77.3 | [Link](configs/mot/bytetrack/detector/yolox_x_24e_800x1440_mix_det.yml) | [Download](https://paddledet.bj.bcebos.com/models/mot/deepsort/yolox_x_24e_800x1440_mix_det.pdparams) |
+| JDE | JDE multi-object tracking algorithm multi-task learning | Edge-Cloud end | MOT-16 test: 64.6 | [Link](configs/mot/jde/jde_darknet53_30e_1088x608.yml) | [Download](https://paddledet.bj.bcebos.com/models/mot/jde_darknet53_30e_1088x608.pdparams) |
+| FairMOT | JDE multi-object tracking algorithm multi-task learning | Edge-Cloud end | MOT-16 test: 75.0 | [Link](configs/mot/fairmot/fairmot_dla34_30e_1088x608.yml) | [Download](https://paddledet.bj.bcebos.com/models/mot/fairmot_dla34_30e_1088x608.pdparams) |
-### Advanced Tutorials
+#### Other multi-object tracking models [docs](configs/mot)
-- Parameter configuration
- - [Parameter configuration for RCNN model](docs/tutorials/config_annotation/faster_rcnn_r50_fpn_1x_coco_annotation_en.md)
- - [Parameter configuration for PP-YOLO model](docs/tutorials/config_annotation/ppyolo_r50vd_dcn_1x_coco_annotation_en.md)
+
-- Model Compression(Based on [PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim))
- - [Prune/Quant/Distill](configs/slim)
+
+ 5. Industrial real-time pedestrain analysis tool-PP Human
-- Inference and deployment
- - [Export model for inference](deploy/EXPORT_MODEL_en.md)
- - [Paddle Inference](deploy/README_en.md)
- - [Python inference](deploy/python)
- - [C++ inference](deploy/cpp)
- - [Paddle-Lite](deploy/lite)
- - [Paddle Serving](deploy/serving)
- - [Export ONNX model](deploy/EXPORT_ONNX_MODEL_en.md)
- - [Inference benchmark](deploy/BENCHMARK_INFER_en.md)
- - [Exporting to ONNX and using OpenVINO for inference](docs/advanced_tutorials/openvino_inference/README.md)
+| Function \ Model | Obejct detection | Multi- object tracking | Attribute recognition | Keypoint detection | Action recognition | ReID |
+|:------------------------------------ |:-------------------------------------------------------------------------------------- |:-------------------------------------------------------------------------------------- |:-----------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------:|:-----------------------------------------------------------------:|:----------------------------------------------------------------------:|
+| Pedestrian Detection | [✅](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | | | | | |
+| Pedestrian Tracking | | [✅](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | | | | |
+| Attribute Recognition (Image) | [✅](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | | [✅](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | | | |
+| Attribute Recognition (Video) | | [✅](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | | | | |
+| Falling Detection | | [✅](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | | [✅](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | [✅](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | |
+| ReID | | [✅](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | | | | [✅](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) |
+| **Accuracy** | mAP 56.3 | MOTA 72.0 | mA 94.86 | AP 87.1 | AP 96.43 | mAP 98.8 |
+| **T4 TensorRT FP16 Inference speed** | 28.0ms | 33.1ms | Single person 2ms | Single person 2.9ms | Single person 2.7ms | Single person 1.5ms |
+
+
+
+**Click “ ✅ ” to download**
+
+##
Document tutorials
+
+### Introductory tutorials
+
+- [Installation](docs/tutorials/INSTALL_cn.md)
+- [Quick start](docs/tutorials/QUICK_STARTED_cn.md)
+- [Data preparation](docs/tutorials/data/README.md)
+- [Geting Started on PaddleDetection](docs/tutorials/GETTING_STARTED_cn.md)
+- [Customize data training]((docs/tutorials/CustomizeDataTraining.md)
+- [FAQ]((docs/tutorials/FAQ)
+
+### Advanced tutorials
+
+- Configuration
+
+ - [RCNN Configuration](docs/tutorials/config_annotation/faster_rcnn_r50_fpn_1x_coco_annotation.md)
+ - [PP-YOLO Configuration](docs/tutorials/config_annotation/ppyolo_r50vd_dcn_1x_coco_annotation.md)
+
+- Compression based on [PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim)
+
+ - [Pruning/Quantization/Distillation Tutorial](configs/slim)
+
+- [Inference deployment](deploy/README.md)
+
+ - [Export model for inference](deploy/EXPORT_MODEL.md)
+
+ - [Paddle Inference deployment](deploy/README.md)
+
+ - [Inference deployment with Python](deploy/python)
+ - [Inference deployment with C++](deploy/cpp)
+
+ - [Paddle-Lite deployment](deploy/lite)
+
+ - [Paddle Serving deployment](deploy/serving)
+
+ - [ONNX model export](deploy/EXPORT_ONNX_MODEL.md)
+
+ - [Inference benchmark](deploy/BENCHMARK_INFER.md)
- Advanced development
- - [New data augmentations](docs/advanced_tutorials/READER_en.md)
- - [New detection algorithms](docs/advanced_tutorials/MODEL_TECHNICAL_en.md)
-
-
-## Model Zoo
-
-- Universal object detection
- - [Model library and baselines](docs/MODEL_ZOO_cn.md)
- - [PP-YOLO](configs/ppyolo/README.md)
- - [PP-PicoDet](configs/picodet/README.md)
- - [Enhanced Anchor Free model--TTFNet](configs/ttfnet/README_en.md)
- - [Mobile models](static/configs/mobile/README_en.md)
- - [676 classes of object detection](static/docs/featured_model/LARGE_SCALE_DET_MODEL_en.md)
- - [Two-stage practical PSS-Det](configs/rcnn_enhance/README_en.md)
- - [SSLD pretrained models](docs/feature_models/SSLD_PRETRAINED_MODEL_en.md)
-- Universal instance segmentation
- - [SOLOv2](configs/solov2/README.md)
-- Rotation object detection
- - [S2ANet](configs/dota/README_en.md)
-- [Keypoint detection](configs/keypoint)
- - [PP-TinyPose](configs/keypoint/tiny_pose)
- - HigherHRNet
- - HRNet
- - LiteHRNet
-- [Multi-Object Tracking](configs/mot/README.md)
- - [PP-Tracking](deploy/pptracking/README.md)
- - [DeepSORT](configs/mot/deepsort/README.md)
- - [JDE](configs/mot/jde/README.md)
- - [FairMOT](configs/mot/fairmot/README.md)
-- Vertical field
- - [Face detection](configs/face_detection/README_en.md)
- - [Pedestrian detection](configs/pedestrian/README.md)
- - [Vehicle detection](configs/vehicle/README.md)
-- Competition Plan
- - [Objects365 2019 Challenge champion model](static/docs/featured_model/champion_model/CACascadeRCNN_en.md)
- - [Best single model of Open Images 2019-Object Detection](static/docs/featured_model/champion_model/OIDV5_BASELINE_MODEL_en.md)
-
-## Applications
-
-- [Christmas portrait automatic generation tool](static/application/christmas)
-- [Android Fitness Demo](https://github.com/zhiboniu/pose_demo_android)
-
-## Updates
-
-Updates please refer to [change log](docs/CHANGELOG_en.md) for details.
-
-
-## License
-
-PaddleDetection is released under the [Apache 2.0 license](LICENSE).
-
-
-## Contributing
-
-Contributions are highly welcomed and we would really appreciate your feedback!!
-- Thanks [Mandroide](https://github.com/Mandroide) for cleaning the code and unifying some function interface.
-- Thanks [FL77N](https://github.com/FL77N/) for contributing the code of `Sparse-RCNN` model.
-- Thanks [Chen-Song](https://github.com/Chen-Song) for contributing the code of `Swin Faster-RCNN` model.
-- Thanks [yangyudong](https://github.com/yangyudong2020), [hchhtc123](https://github.com/hchhtc123) for contributing PP-Tracking GUI interface.
-- Thanks [Shigure19](https://github.com/Shigure19) for contributing PP-TinyPose fitness APP.
-
-## Citation
+
+ - [Data processing module](docs/advanced_tutorials/READER.md)
+ - [New object detection models](docs/advanced_tutorials/MODEL_TECHNICAL.md)
+ - Custumization
+ - [Object detection](docs/advanced_tutorials/customization/detection.md)
+ - [Keypoint detection](docs/advanced_tutorials/customization/keypoint_detection.md)
+ - [Multiple object tracking](docs/advanced_tutorials/customization/pphuman_mot.md)
+ - [Action recognition](docs/advanced_tutorials/customization/pphuman_action.md)
+ - [Attribute recognition](docs/advanced_tutorials/customization/pphuman_attribute.md)
+
+### Courses
+
+- **[Theoretical foundation] [Object detection 7-day camp](https://aistudio.baidu.com/aistudio/education/group/info/1617):** Overview of object detection tasks, details of RCNN series object detection algorithm and YOLO series object detection algorithm, PP-YOLO optimization strategy and case sharing, introduction and practice of AnchorFree series algorithm
+
+- **[Industrial application] [AI Fast Track industrial object detection technology and application](https://aistudio.baidu.com/aistudio/education/group/info/23670):** Super object detection algorithms, real-time pedestrian analysis system PP-Human, breakdown and practice of object detection industrial application
+
+- **[Industrial features] 2022.3.26** **[Smart City Industry Seven-Day Class](https://aistudio.baidu.com/aistudio/education/group/info/25620)** : Urban planning, Urban governance, Smart governance service, Traffic management, community governance.
+
+### [Industrial tutorial examples](./industrial_tutorial/README.md)
+
+- [Intelligent fitness recognition based on PP-TinyPose Plus](https://aistudio.baidu.com/aistudio/projectdetail/4385813)
+
+- [Road litter detection based on PP-PicoDet Plus](https://aistudio.baidu.com/aistudio/projectdetail/3561097)
+
+- [Communication tower detection based on PP-PicoDet and deployment on Android](https://aistudio.baidu.com/aistudio/projectdetail/3561097)
+
+- [Visitor flow statistics based on FairMOT](https://aistudio.baidu.com/aistudio/projectdetail/2421822)
+
+- [More examples](./industrial_tutorial/README.md)
+
+##
Applications
+
+- [Fitness app on android mobile](https://github.com/zhiboniu/pose_demo_android)
+- [PP-Tracking GUI Visualization Interface](https://github.com/yangyudong2020/PP-Tracking_GUi)
+
+## Recommended third-party tutorials
+
+- [Deployment of PaddleDetection for Windows I ](https://zhuanlan.zhihu.com/p/268657833)
+- [Deployment of PaddleDetection for Windows II](https://zhuanlan.zhihu.com/p/280206376)
+- [Deployment of PaddleDetection on Jestson Nano](https://zhuanlan.zhihu.com/p/319371293)
+- [How to deploy YOLOv3 model on Raspberry Pi for Helmet detection](https://github.com/PaddleCV-FAQ/PaddleDetection-FAQ/blob/main/Lite%E9%83%A8%E7%BD%B2/yolov3_for_raspi.md)
+- [Use SSD-MobileNetv1 for a project -- From dataset to deployment on Raspberry Pi](https://github.com/PaddleCV-FAQ/PaddleDetection-FAQ/blob/main/Lite%E9%83%A8%E7%BD%B2/ssd_mobilenet_v1_for_raspi.md)
+
+##
Version updates
+
+Please refer to the[ Release note ](https://github.com/PaddlePaddle/Paddle/wiki/PaddlePaddle-2.3.0-Release-Note-EN)for more details about the updates
+
+##
License
+
+PaddlePaddle is provided under the [Apache 2.0 license](LICENSE)
+
+##
Contribute your code
+
+We appreciate your contributions and your feedback！
+
+- Thank [Mandroide](https://github.com/Mandroide) for code cleanup and
+- Thank [FL77N](https://github.com/FL77N/) for `Sparse-RCNN`model
+- Thank [Chen-Song](https://github.com/Chen-Song) for `Swin Faster-RCNN`model
+- Thank [yangyudong](https://github.com/yangyudong2020), [hchhtc123](https://github.com/hchhtc123) for developing PP-Tracking GUI interface
+- Thank Shigure19 for developing PP-TinyPose fitness APP
+- Thank [manangoel99](https://github.com/manangoel99) for Wandb visualization methods
+
+##
Quote
```
@misc{ppdet2019,
diff --git "a/activity/\347\233\264\346\222\255\347\255\224\347\226\221\347\254\254\344\270\200\346\234\237.md" "b/activity/\347\233\264\346\222\255\347\255\224\347\226\221\347\254\254\344\270\200\346\234\237.md"
new file mode 100644
index 0000000000000000000000000000000000000000..f94f0dd09941474558bf9dc6baac0669c8ded9c3
--- /dev/null
+++ "b/activity/\347\233\264\346\222\255\347\255\224\347\226\221\347\254\254\344\270\200\346\234\237.md"
@@ -0,0 +1,125 @@
+# 直播答疑第一期
+
+### 答疑全程回放可以通过链接下载观看：https://pan.baidu.com/s/168ouju4MxN5XJEb-GU1iAw 提取码: 92mw
+
+## PaddleDetection框架/API问题
+
+#### Q1. warmup能详细讲解下吗？
+A1. warmup是在训练初期学习率从0调整至预设学习率的过程，设置可以参考[源码](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/ppdet/optimizer.py#L156)，可以设置step数或epoch数
+
+#### Q2. 如果类别不匹配 也能用pretrain weights吗？
+A2. 可以，类别不匹配时，模型会自动不加载shape不匹配的权重，通常和类别数相关的权重位于head层
+
+#### Q3. 请问nms_eta怎么用呀，源码上没有写的很清楚，API文档也没有细说
+A3. 针对密集的场景，nms_eta会在每轮动态的调整nms阈值，避免过滤掉两个重叠程度很高但是属于不同物体的检测框，具体可以参考[源码](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/detection/multiclass_nms_op.cc#L139)，默认为1，通常无需设置
+
+#### Q4. 请问anchor_cluster.py中的--size 是模型的input size 还是 实际使用图片的size？
+A4. 是实际推理时的图片尺寸，一般可以参考TestReader中的image_shape的设置。
+
+#### Q5. 请问为什么预测的坐标会出现负的值？
+A5. 模型算法中是有可能负值的情况，首先需要判断模型预测效果是否符合预期，如果正常可以考虑在后处理中增加clip的操作限制输出box在图像中；如果不正常，说明模型训练效果欠佳，需要进一步排查问题或调优
+
+#### Q6. PaddleDetection 人脸检测blazeface模型，一键式预测时load_params没有参数文件，从哪里下载?
+A6. blazeface的模型可以在[模型库](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.4/configs/face_detection#%E6%A8%A1%E5%9E%8B%E5%BA%93)中下载到，如果想部署需要参考[步骤](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/deploy/EXPORT_MODEL.md) 导出模型
+
+## PP-YOLOE问题
+#### Q1. 训练PP-YOLOE的时候，loss是越训练越高这种情况 是数据集的问题吗？
+A1. 可以从以下几个方面排查
+
+1. 数据: 首先确认数据集没问题，包括标注，类别等
+2. 超参数：base_lr根据batch_size调整，遵守线性原则；warmup_iters根据总的epoch数进行调整
+3. 预训练参数：可以加载官方提供的自在coco数据集上的预训练参数
+4. 网络结构方面：分析下box的分布情况 适当调整dfl的参数
+
+#### Q2. 检测模型选型问题：PicoDet、PP-YOLO系列如何选型
+A2. PicoDet是针对移动端设备设计的模型，是针对arm，x86等低算力设备上设计；PP-YOLO是针对服务器端设计的模型，英伟达N卡，百度昆仑卡等。手机端，无gpu桌面端，优先PicoDet；有高算力设备，如N卡，优先PP-YOLO系列；对延时不敏感的场景，更注重高精度，优先PP-YOLO系列
+
+#### Q3. ConvBNLayer中BN层的参数都不会使用L2Decay；PP-YOLOE-s的其它部分都会按照配置文件的设置使用0.0005的L2Decay。是这样吗
+A3. PP-YOLOE的backbone和neck部分使用了ConvBNLayer，其中BN层不会使用L2Decay，其他部分使用全局设置的0.0005的L2Decay
+
+#### Q4. PP-YOLOE的Conv的bias也不使用decay吗？
+A4. PP-YOLOE的backbone和neck部分的Conv是没有bias参数的，head部分的Conv bias使用全局decay
+
+#### Q5. 在测速时，为什么要用PaddleInference而不是直接加载模型测时间呢
+A5. PaddleInference会将paddle导出的预测模型会前向算子做融合，从而实现速度优化，并且实际部署过程也是使用PaddleInference实现
+
+#### Q6. PP-YOLOE系列在部署的时候，前后处理是不是一样的啊？
+A6. PP-YOLO系列模型在部署时的前处理都是 decode-resize-nomalize-permute的流程，后处理方面PP-YOLOv2使用了Matrix NMS，PP-YOLOE使用的是普通的NMS算法
+
+#### Q7. 针对小目标和类别不平衡的数据集，PP-YOLOE有什么调整策略吗
+A7 针对小目标数据集，可以适当增大ppyoloe的输入尺寸，然后在模型中增加注意力机制，目前基于PP-YOLOE的小目标检测正在开发中；针对类别不平衡问题，可以从数据采样的角度处理，目前PP-YOLOE还没有专门针对类别不平衡问题的优化
+
+## PP-Human问题
+#### Q1. 请问pphuman用导出的模型18个点（不是官方17个点）去预测时，报错是问什么
+A1. 这个问题是关键点模型输出点的数量与行为识别模型不一致导致的。如果希望用18点模型预测，除了关键点用18点模型以外，还需要自建18点的动作识别模型。
+
+#### Q2. 为什么官方导出模型设置的window_size是50
+A2. 导出模型的设置与训练和预测的输入数据长度是一致的；我们主要采用的数据集是ntu、企业提供的实际数据等等。在训练这个模型的时候，我们对这些数据中摔倒的片段做了统计分析，基本上每个动作片段持续的帧数大约是40~80左右。综合考虑到实际使用的延迟以及预测效果，我们选择了50这个量级，在我们的这部分数据上既能完整描述一个完整动作，又不会使得延迟过大。
+
+总的来说，这个window_size的数值最好还是根据实际动作以及设备的情况进行选择。例如在某种设备上，50帧的长度根本不足以包含一个完整的动作，那么这个数值就需要扩大；又或者某些动作持续时间很短，50帧的长度包含了太多不相关的其他动作，容易造成误识别，那么这个数值可以适当缩小。
+
+
+#### Q3. PP-Human中如何替换检测、跟踪、关键点模型
+A3. 我们使用的模型都是PaddleDetection中模型进行导出得到的。理论上PP-Human所使用的模型都是可以直接替换的，但是需要注意是流程和前后处理一样的模型。
+
+#### Q4. PP-Human中的数据标注问题（检测、跟踪、关键点、行为、属性）标注工具推荐和标注步骤
+A4. 标注工具：检测 labelme, labelImg, cvat； 跟踪darklabel，cvat；关键点 labelme，cvat。检测标注可以使用tools/x2coco.py转换成coco格式
+
+#### Q5. PP-Human中如何更改label（属性和动作识别）
+A5. 在PPHuman中，动作识别被定义为基于骨骼点序列的分类问题，目前我们已经开源的摔倒动作识别是一个二分类问题；属性方面我们当前还暂时没有开放训练，正在建设中
+
+#### Q6. PP-Human的哪些功能支持单人、哪些支持多人
+A6. PP-Human的功能实现基于一套流程：检测->跟踪->具体功能。当前我们的具体功能模型每次处理的是单人的，即属性、动作等都是属于图像中每一个具体人的。但是基于这套流程下来，图像中的每一个人都得到了处理的。所以单人、多人实际都是一样支持的。
+
+#### Q7. PP-Human对视频流预测的支持及服务化部署
+A7. 目前正在建设当中，下个版本会支持这部分功能
+
+#### Q8. 在使用pphuman训练自己的数据集时，训练完进行测试时，可视化的标签如何更改，没有更改的情况下还是falling
+
+A8. 可视化的函数位于https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/deploy/python/visualize.py#L368，这里在可视化的时候将 action_text替换为期望的类别即可。
+
+#### Q9. 关键点检测可以实现一个连贯动作的检测吗，比如健身规范
+A9. 基于关键点是可以实现的。这里可以有不同思路去做：
+
+1. 如果是期望判定动作规范的程度，且这个动作可以很好的描述。那么可以在关键点模型获得的坐标的基础上，人工增加逻辑判断即可。这里我们提供一个安卓的健身APP示例：https://github.com/zhiboniu/pose_demo_android ，其中实现判定各项动作的逻辑可以参考https://github.com/zhiboniu/pose_demo_android/blob/release/1.0/app/src/main/cpp/pose_action.cc 。
+
+2. 当一个动作较难用逻辑去描述的时候，可能参考现有摔倒检测的案例，训练一个识别健身动作的模型，但对收集数据的要求会比较高。
+
+
+#### Q10. 有遮挡的生产环境中梯子，可以用关键点检测判断人员上下梯动作是否合规
+A10. 这个问题需要视遮挡的程度而定，如果遮挡过于严重时关键点检测模型的效果会大打折扣，从而导致行为的判断失准。此外，由于基于关键点的方案抹去了外观信息，如果只是从人物本身的动作上去做判断，那么在遮挡不严重的场景下是可以的。反之，如果梯子这个物体是判断动作是否合规的必要元素，那么这个方案其实不一定是最佳选择。
+
+#### Q11. 关键点做的行为识别并不是时序上的动作识别吗
+A11. 是时序的动作识别。这里是将一定时间范围内的每一帧关键点坐标组成一个时序的关键点序列，再通过行为识别模型去预测这个序列所属的行为类别。
+
+
+## 检测算法问题
+#### Q1. 大图片小目标 最终推理的图片也是大图片 怎么预处理呀
+A1. 小目标问题常见的处理方式是切图以及增大网络输入尺寸，如果使用基于anchor的检测算法，可以通过对目标物体大小聚类生成anchor，参考[脚本](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/tools/anchor_cluster.py)； 目前基于PP-YOLOE的小目标检测正在开发中
+
+#### Q2. 想问下大的目标对象怎么检测，比如发票
+A2. 如果使用基于anchor的检测算法，可以通过对目标物体大小聚类生成anchor，参考[脚本](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/tools/anchor_cluster.py)；另外可以增强深层特征提升大物体检测效果
+
+#### Q3. 在做预测时发现预测框特别多，有的框的置信度甚至低于0.1，请问如果将这种框过滤掉？也就是训练模型时就把这些极低置信度的预测结果过滤掉，避免在推理部署时，做不必要的计算，从而影响推理速度。
+A3. 后处理部分有两个过滤，1）是提取置信度最高的Top 100个框做nms。2）是根据设定阈值threshold进行过滤。如果你可以确认图片上目标相对比较少<10个，可以调整Top 100这个值到50或者更低，这样可以加速nms部分的计算。其次调整threshold这个影响最终检测的准确度和召回率的效果。
+
+#### Q4. 正负样本的比例一般怎么设计
+A4. 在PaddleDetection中，支持负样本训练，TrainDataset下设置allow_empty: true即可，通过数据集测试，负样本比例在0.3时对模型提升效果最明显。
+
+## 压缩部署问题
+#### Q1. PaddleDetection训练的模型导出inference model后，在做推理部署的时候，前后处理相关代码如何编写，有什么参考教程吗？
+A1. 目前PaddleDetection下的网络模型大部分都能够支持c++ inference，不同的处理方式针对不同功能，例如：PP-YOLOE速度测试不包含后处理，PicoDet为支持不同的第三方推理引擎会设置是否导出nms
+
+object_detector.cc是针对所有检测模型的流程，其中前处理大部分都是decode-resize-nomalize-permute 部分网络会加入padding的操作；大部分模型的后处理操作都放在模型里面了，picodet有单独提供nms的后处理代码
+
+检测模型的输入统一为image，im_shape，scale_factor ，如果模型中没有使用im_shape，输出个数会减少，但是整套预处理流程不需要额外开发
+
+#### Q2. 针对TensorRT的加速问题，fp16在v100确实可以，但是耗时好像有点偏差，我在1080ti上，单张图片跑1000次，耗时50s，还是float32的，可是在v100上，float16耗时97
+A2. 目前PPYOLOE等模型的速度都有在V100上使用TensorRT FP16测试，关于速度测试有以下几个方面可以排查：
+
+1. 速度测试时是否正确设置warmup，以避免过长的启动时间影响速度测试准确度
+2. 在开启TensorRT时，生成engine文件的过程耗时较长，可以在https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/deploy/python/infer.py#L745 中将use_static设置为True
+
+
+#### Q3. PaddleDetection已经支持了在线量化一些模型，比如想训练其他的一个新模型，是不是可以轻松用起来qat？如果不能，为什么只能支持很有限的模型，而qat其他模型总会出各种各样的问题，原因是什么？
+A3. 目前PaddleDetection模型很多，只能针对部分模型开源了QAT的config，其他模型也是支持QAT的，只是配置文件没有覆盖到，如果量化报错，通常是配置问题。检测模型一般建议跳过head最后一个conv。如果想要跳过某些层量化，可以设置skip_quant，参考[代码](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/ppdet/modeling/heads/yolo_head.py#L97)
diff --git a/configs/centernet/_base_/centernet_reader.yml b/configs/centernet/_base_/centernet_reader.yml
index 1f18dca49d1e39c61b9bbc7b5be8bfac7bce5ca4..81af4ab840502da6e738ac667dd0883041ba8992 100644
--- a/configs/centernet/_base_/centernet_reader.yml
+++ b/configs/centernet/_base_/centernet_reader.yml
@@ -30,6 +30,6 @@ TestReader:
sample_transforms:
- Decode: {}
- WarpAffine: {keep_res: True, input_h: 512, input_w: 512}
- - NormalizeImage: {mean: [0.40789655, 0.44719303, 0.47026116], std: [0.2886383 , 0.27408165, 0.27809834]}
+ - NormalizeImage: {mean: [0.40789655, 0.44719303, 0.47026116], std: [0.2886383 , 0.27408165, 0.27809834], is_scale: True}
- Permute: {}
batch_size: 1
diff --git a/configs/convnext/README.md b/configs/convnext/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..644d66815660427d2a6cdf587c014d8cb877eb15
--- /dev/null
+++ b/configs/convnext/README.md
@@ -0,0 +1,20 @@
+# ConvNeXt (A ConvNet for the 2020s)
+
+## 模型库
+### ConvNeXt on COCO
+
+| 网络网络 | 输入尺寸 | 图片数/GPU | 学习率策略 | mAP
val
0.5:0.95 | mAPval
0.5 | Params(M) | FLOPs(G) | 下载链接 | 配置文件 |
+| :------------- | :------- | :-------: | :------: | :------------: | :---------------------: | :----------------: |:---------: | :------: |:---------------: |
+| PP-YOLOE-ConvNeXt-tiny | 640 | 16 | 36e | 44.6 | 63.3 | 33.04 | 13.87 | [下载链接](https://paddledet.bj.bcebos.com/models/ppyoloe_convnext_tiny_36e_coco.pdparams) | [配置文件](./ppyoloe_convnext_tiny_36e_coco.yml) |
+| YOLOX-ConvNeXt-s | 640 | 8 | 36e | 44.6 | 65.3 | 36.20 | 27.52 | [下载链接](https://paddledet.bj.bcebos.com/models/yolox_convnext_s_36e_coco.pdparams) | [配置文件](./yolox_convnext_s_36e_coco.yml) |
+
+
+## Citations
+```
+@Article{liu2022convnet,
+ author = {Zhuang Liu and Hanzi Mao and Chao-Yuan Wu and Christoph Feichtenhofer and Trevor Darrell and Saining Xie},
+ title = {A ConvNet for the 2020s},
+ journal = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
+ year = {2022},
+}
+```
diff --git a/configs/convnext/ppyoloe_convnext_tiny_36e_coco.yml b/configs/convnext/ppyoloe_convnext_tiny_36e_coco.yml
new file mode 100644
index 0000000000000000000000000000000000000000..360a368ec0837033ab408db59aa0d4ea5b7972dd
--- /dev/null
+++ b/configs/convnext/ppyoloe_convnext_tiny_36e_coco.yml
@@ -0,0 +1,55 @@
+_BASE_: [
+ '../datasets/coco_detection.yml',
+ '../runtime.yml',
+ '../ppyoloe/_base_/ppyoloe_crn.yml',
+ '../ppyoloe/_base_/ppyoloe_reader.yml',
+]
+depth_mult: 0.25
+width_mult: 0.50
+
+log_iter: 100
+snapshot_epoch: 5
+weights: output/ppyoloe_convnext_tiny_36e_coco/model_final
+pretrain_weights: https://bj.bcebos.com/v1/paddledet/models/pretrained/convnext_tiny_22k_224.pdparams
+
+
+YOLOv3:
+ backbone: ConvNeXt
+ neck: CustomCSPPAN
+ yolo_head: PPYOLOEHead
+ post_process: ~
+
+ConvNeXt:
+ arch: 'tiny'
+ drop_path_rate: 0.4
+ layer_scale_init_value: 1.0
+ return_idx: [1, 2, 3]
+
+
+PPYOLOEHead:
+ static_assigner_epoch: 12
+ nms:
+ nms_top_k: 10000
+ keep_top_k: 300
+ score_threshold: 0.01
+ nms_threshold: 0.7
+
+
+TrainReader:
+ batch_size: 16
+
+
+epoch: 36
+LearningRate:
+ base_lr: 0.0002
+ schedulers:
+ - !PiecewiseDecay
+ gamma: 0.1
+ milestones: [36]
+ use_warmup: false
+
+OptimizerBuilder:
+ regularizer: false
+ optimizer:
+ type: AdamW
+ weight_decay: 0.0005
diff --git a/configs/convnext/yolox_convnext_s_36e_coco.yml b/configs/convnext/yolox_convnext_s_36e_coco.yml
new file mode 100644
index 0000000000000000000000000000000000000000..b41551dee8a2e2793ac09d474c0e7d2a8868299f
--- /dev/null
+++ b/configs/convnext/yolox_convnext_s_36e_coco.yml
@@ -0,0 +1,58 @@
+_BASE_: [
+ '../datasets/coco_detection.yml',
+ '../runtime.yml',
+ '../yolox/_base_/yolox_cspdarknet.yml',
+ '../yolox/_base_/yolox_reader.yml'
+]
+depth_mult: 0.33
+width_mult: 0.50
+
+log_iter: 100
+snapshot_epoch: 5
+weights: output/yolox_convnext_s_36e_coco/model_final
+pretrain_weights: https://bj.bcebos.com/v1/paddledet/models/pretrained/convnext_tiny_22k_224.pdparams
+
+
+YOLOX:
+ backbone: ConvNeXt
+ neck: YOLOCSPPAN
+ head: YOLOXHead
+ size_stride: 32
+ size_range: [15, 25] # multi-scale range [480*480 ~ 800*800]
+
+ConvNeXt:
+ arch: 'tiny'
+ drop_path_rate: 0.4
+ layer_scale_init_value: 1.0
+ return_idx: [1, 2, 3]
+
+
+TrainReader:
+ batch_size: 8
+ mosaic_epoch: 30
+
+
+YOLOXHead:
+ l1_epoch: 30
+ nms:
+ name: MultiClassNMS
+ nms_top_k: 10000
+ keep_top_k: 1000
+ score_threshold: 0.001
+ nms_threshold: 0.65
+
+
+epoch: 36
+LearningRate:
+ base_lr: 0.0002
+ schedulers:
+ - !PiecewiseDecay
+ gamma: 0.1
+ milestones: [36]
+ use_warmup: false
+
+OptimizerBuilder:
+ regularizer: false
+ optimizer:
+ type: AdamW
+ weight_decay: 0.0005
diff --git a/configs/datasets/coco_detection.yml b/configs/datasets/coco_detection.yml
index 7a62c3b0b57a5d76c8ed519d3a3940c1b4532c15..291c24874b72bbb92fb2510e754c791a3f06c146 100644
--- a/configs/datasets/coco_detection.yml
+++ b/configs/datasets/coco_detection.yml
@@ -16,4 +16,5 @@ EvalDataset:
TestDataset:
!ImageFolder
- anno_path: annotations/instances_val2017.json
+ anno_path: annotations/instances_val2017.json # also support txt (like VOC's label_list.txt)
+ dataset_dir: dataset/coco # if set, anno_path will be 'dataset_dir/anno_path'
diff --git a/configs/datasets/coco_instance.yml b/configs/datasets/coco_instance.yml
index 5eaf76791a94bfd2819ba6dab610fae54b69f26e..b04dbdca955a326ffc5eb13756e73ced83b92309 100644
--- a/configs/datasets/coco_instance.yml
+++ b/configs/datasets/coco_instance.yml
@@ -16,4 +16,5 @@ EvalDataset:
TestDataset:
!ImageFolder
- anno_path: annotations/instances_val2017.json
+ anno_path: annotations/instances_val2017.json # also support txt (like VOC's label_list.txt)
+ dataset_dir: dataset/coco # if set, anno_path will be 'dataset_dir/anno_path'
diff --git a/configs/datasets/dota.yml b/configs/datasets/dota.yml
index f9d9395b00d7ed9028396044c407784d251e43e5..5153163d95a8a418a82d3d6d43f6e1f9404ed075 100644
--- a/configs/datasets/dota.yml
+++ b/configs/datasets/dota.yml
@@ -17,3 +17,4 @@ EvalDataset:
TestDataset:
!ImageFolder
anno_path: trainval_split/s2anet_trainval_paddle_coco.json
+ dataset_dir: dataset/DOTA_1024_s2anet/
diff --git a/configs/datasets/roadsign_voc.yml b/configs/datasets/roadsign_voc.yml
index ddbfc7889e0027d85971c6ab11f3f33adfe8be71..9a081611aa8dafef5d5c6f1af1476cc038db5702 100644
--- a/configs/datasets/roadsign_voc.yml
+++ b/configs/datasets/roadsign_voc.yml
@@ -3,19 +3,19 @@ map_type: integral
num_classes: 4
TrainDataset:
- !VOCDataSet
- dataset_dir: dataset/roadsign_voc
- anno_path: train.txt
- label_list: label_list.txt
- data_fields: ['image', 'gt_bbox', 'gt_class', 'difficult']
+ name: VOCDataSet
+ dataset_dir: dataset/roadsign_voc
+ anno_path: train.txt
+ label_list: label_list.txt
+ data_fields: ['image', 'gt_bbox', 'gt_class', 'difficult']
EvalDataset:
- !VOCDataSet
- dataset_dir: dataset/roadsign_voc
- anno_path: valid.txt
- label_list: label_list.txt
- data_fields: ['image', 'gt_bbox', 'gt_class', 'difficult']
+ name: VOCDataSet
+ dataset_dir: dataset/roadsign_voc
+ anno_path: valid.txt
+ label_list: label_list.txt
+ data_fields: ['image', 'gt_bbox', 'gt_class', 'difficult']
TestDataset:
- !ImageFolder
- anno_path: dataset/roadsign_voc/label_list.txt
+ name: ImageFolder
+ anno_path: dataset/roadsign_voc/label_list.txt
diff --git a/configs/datasets/visdrone_detection.yml b/configs/datasets/visdrone_detection.yml
new file mode 100644
index 0000000000000000000000000000000000000000..37feb6e2618ff9d83ce2842a9e581dcfd31efc78
--- /dev/null
+++ b/configs/datasets/visdrone_detection.yml
@@ -0,0 +1,22 @@
+metric: COCO
+num_classes: 10
+
+TrainDataset:
+ !COCODataSet
+ image_dir: VisDrone2019-DET-train
+ anno_path: train.json
+ dataset_dir: dataset/visdrone
+ data_fields: ['image', 'gt_bbox', 'gt_class', 'is_crowd']
+
+EvalDataset:
+ !COCODataSet
+ image_dir: VisDrone2019-DET-val
+ anno_path: val.json
+ # image_dir: test_dev
+ # anno_path: test_dev.json
+ dataset_dir: dataset/visdrone
+
+TestDataset:
+ !ImageFolder
+ anno_path: val.json
+ dataset_dir: dataset/visdrone
diff --git a/configs/dota/README.md b/configs/dota/README.md
index 9a5988a761810600b02cb4e9f1348c6072e02cac..adde27691ef78606d2528b95dee8e30842bfff64 100644
--- a/configs/dota/README.md
+++ b/configs/dota/README.md
@@ -53,7 +53,7 @@ DOTA数据集中总共有2806张图像，其中1411张图像作为训练集，45
- PaddlePaddle >= 2.1.1
- GCC == 8.2
-推荐使用docker镜像[paddle:2.1.1-gpu-cuda10.1-cudnn7](registry.baidubce.com/paddlepaddle/paddle:2.1.1-gpu-cuda10.1-cudnn7)。
+推荐使用docker镜像 paddle:2.1.1-gpu-cuda10.1-cudnn7。
执行如下命令下载镜像并启动容器：
```
diff --git a/configs/dota/README_en.md b/configs/dota/README_en.md
index e299e0e81808888e947d0e0b1e1423bb5f7fdbea..61eeee7f5c53b7ec4e01c2a68c75f98f9a09bd14 100644
--- a/configs/dota/README_en.md
+++ b/configs/dota/README_en.md
@@ -64,7 +64,7 @@ To use the rotating frame IOU to calculate the OP, the following conditions must
- PaddlePaddle >= 2.1.1
- GCC == 8.2
-Docker images are recommended[paddle:2.1.1-gpu-cuda10.1-cudnn7](registry.baidubce.com/paddlepaddle/paddle:2.1.1-gpu-cuda10.1-cudnn7)。
+Docker images are recommended paddle:2.1.1-gpu-cuda10.1-cudnn7。
Run the following command to download the image and start the container:
```
diff --git a/configs/faster_rcnn/_base_/faster_rcnn_swin_reader.yml b/configs/faster_rcnn/_base_/faster_rcnn_swin_reader.yml
index 396462a2ff7cf68f7d91a1bb1bb87b8e5a040486..1af6175a931a571f1c6726f0f312591c07489d1d 100644
--- a/configs/faster_rcnn/_base_/faster_rcnn_swin_reader.yml
+++ b/configs/faster_rcnn/_base_/faster_rcnn_swin_reader.yml
@@ -30,14 +30,12 @@ EvalReader:
TestReader:
inputs_def:
- image_shape: [1, 3, 640, 640]
+ image_shape: [-1, 3, 640, 640]
sample_transforms:
- Decode: {}
- - Resize: {interp: 2, target_size: [640, 640], keep_ratio: True}
+ - LetterBoxResize: {target_size: 640}
- NormalizeImage: {is_scale: true, mean: [0.485,0.456,0.406], std: [0.229, 0.224,0.225]}
- Permute: {}
- batch_transforms:
- - PadBatch: {pad_to_stride: 32}
batch_size: 1
shuffle: false
drop_last: false
diff --git a/configs/fcos/_base_/fcos_r50_fpn.yml b/configs/fcos/_base_/fcos_r50_fpn.yml
index 64a275d88023030b2299b0c3932b1c3fc9ce1e34..cd22c229a1192ea384037848be1b7e6edc43741a 100644
--- a/configs/fcos/_base_/fcos_r50_fpn.yml
+++ b/configs/fcos/_base_/fcos_r50_fpn.yml
@@ -30,7 +30,6 @@ FCOSHead:
num_convs: 4
norm_type: "gn"
use_dcn: false
- num_classes: 80
fpn_stride: [8, 16, 32, 64, 128]
prior_prob: 0.01
fcos_loss: FCOSLoss
@@ -46,7 +45,6 @@ FCOSLoss:
FCOSPostProcess:
decode:
name: FCOSBox
- num_classes: 80
nms:
name: MultiClassNMS
nms_top_k: 1000
diff --git a/configs/keypoint/README.md b/configs/keypoint/README.md
index e750312a0f0c17197ca74032d00d97978298549d..4ca9b07474caca53ca529fc19bd5b239acb742e4 100644
--- a/configs/keypoint/README.md
+++ b/configs/keypoint/README.md
@@ -1,66 +1,110 @@
简体中文 | [English](README_en.md)
-# KeyPoint模型系列
+# 关键点检测系列模型
+
+
+
-
-
关键点：128x96 | 检测mAP：29.0
关键点AP：58.1 | 检测耗时：2.37ms
关键点耗时：3.27ms | 检测：1.18
关键点：1.36 | 检测：0.35
关键点：0.08 | [检测](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian.pdparams)
[关键点](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96.pdparams) | [检测](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian_fp16.nb)
[关键点](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96_fp16.nb) |
+| [PicoDet-S-Pedestrian](../picodet/legacy_model/application/pedestrian_detection/picodet_s_320_pedestrian.yml) | [PP-TinyPose](./tiny_pose/tinypose_256x192.yml) | 检测：320x320
关键点：256x192 | 检测mAP：38.5
关键点AP：68.8 | 检测耗时：6.30ms
关键点耗时：8.33ms | 检测：1.18
关键点：1.36 | 检测：0.97
关键点：0.32 | [检测](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian.pdparams)
[关键点](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192.pdparams) | [检测](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian_fp16.nb)
[关键点](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192_fp16.nb) |
+
+
+*详细关于PP-TinyPose的使用请参考[文档](./tiny_pose/README.md)。
+
+### 服务端模型推荐
+
+| 检测模型 | 关键点模型 | 输入尺寸 | COCO数据集精度 | 参数量 （M） | Flops (G) | 模型权重 |
+| :----------------------------------------------------------- | :----------------------------------------- | :------------------------------: | :-----------------------------: | :----------------------: | :----------------------: | :----------------------------------------------------------: |
+| [PP-YOLOv2](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [HRNet-w32](./hrnet/hrnet_w32_384x288.yml) | 检测：640x640
关键点：384x288 | 检测mAP：49.5
关键点AP：77.8 | 检测：54.6
关键点：28.6 | 检测：115.8
关键点：17.3 | [检测](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams)
[关键点](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams) |
+| [PP-YOLOv2](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [HRNet-w32](./hrnet/hrnet_w32_256x192.yml) | 检测：640x640
关键点：256x192 | 检测mAP：49.5
关键点AP：76.9 | 检测：54.6
关键点：28.6 | 检测：115.8
关键点：7.68 | [检测](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams)
[关键点](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_384x288.pdparams) |
+
+
+## 模型库
+COCO数据集
+| 模型 | 方案 |输入尺寸 | AP(coco val) | 模型下载 | 配置文件 |
+| :---------------- | -------- | :----------: | :----------------------------------------------------------: | ----------------------------------------------------| ------- |
+| HigherHRNet-w32 |Bottom-Up| 512 | 67.1 | [higherhrnet_hrnet_w32_512.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/higherhrnet_hrnet_w32_512.pdparams) | [config](./higherhrnet/higherhrnet_hrnet_w32_512.yml) |
+| HigherHRNet-w32 | Bottom-Up| 640 | 68.3 | [higherhrnet_hrnet_w32_640.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/higherhrnet_hrnet_w32_640.pdparams) | [config](./higherhrnet/higherhrnet_hrnet_w32_640.yml) |
+| HigherHRNet-w32+SWAHR |Bottom-Up| 512 | 68.9 | [higherhrnet_hrnet_w32_512_swahr.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/higherhrnet_hrnet_w32_512_swahr.pdparams) | [config](./higherhrnet/higherhrnet_hrnet_w32_512_swahr.yml) |
+| HRNet-w32 | Top-Down| 256x192 | 76.9 | [hrnet_w32_256x192.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams) | [config](./hrnet/hrnet_w32_256x192.yml) |
+| HRNet-w32 |Top-Down| 384x288 | 77.8 | [hrnet_w32_384x288.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_384x288.pdparams) | [config](./hrnet/hrnet_w32_384x288.yml) |
+| HRNet-w32+DarkPose |Top-Down| 256x192 | 78.0 | [dark_hrnet_w32_256x192.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/dark_hrnet_w32_256x192.pdparams) | [config](./hrnet/dark_hrnet_w32_256x192.yml) |
+| HRNet-w32+DarkPose |Top-Down| 384x288 | 78.3 | [dark_hrnet_w32_384x288.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/dark_hrnet_w32_384x288.pdparams) | [config](./hrnet/dark_hrnet_w32_384x288.yml) |
+| WiderNaiveHRNet-18 | Top-Down|256x192 | 67.6(+DARK 68.4) | [wider_naive_hrnet_18_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/wider_naive_hrnet_18_256x192_coco.pdparams) | [config](./lite_hrnet/wider_naive_hrnet_18_256x192_coco.yml) |
+| LiteHRNet-18 |Top-Down| 256x192 | 66.5 | [lite_hrnet_18_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_18_256x192_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_18_256x192_coco.yml) |
+| LiteHRNet-18 |Top-Down| 384x288 | 69.7 | [lite_hrnet_18_384x288_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_18_384x288_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_18_384x288_coco.yml) |
+| LiteHRNet-30 | Top-Down|256x192 | 69.4 | [lite_hrnet_30_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_30_256x192_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_30_256x192_coco.yml) |
+| LiteHRNet-30 |Top-Down| 384x288 | 72.5 | [lite_hrnet_30_384x288_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_30_384x288_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_30_384x288_coco.yml) |
备注： Top-Down模型测试AP结果基于GroundTruth标注框
MPII数据集
-| 模型 | 输入尺寸 | PCKh(Mean) | PCKh(Mean@0.1) | 模型下载 | 配置文件 |
-| :---- | -------- | :--------: | :------------: | :----------------------------------------------------------: | -------------------------------------------- |
-| HRNet-w32 | 256x256 | 90.6 | 38.5 | [hrnet_w32_256x256_mpii.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x256_mpii.pdparams) | [config](./hrnet/hrnet_w32_256x256_mpii.yml) |
+| 模型 | 方案| 输入尺寸 | PCKh(Mean) | PCKh(Mean@0.1) | 模型下载 | 配置文件 |
+| :---- | ---|----- | :--------: | :------------: | :----------------------------------------------------------: | -------------------------------------------- |
+| HRNet-w32 | Top-Down|256x256 | 90.6 | 38.5 | [hrnet_w32_256x256_mpii.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x256_mpii.pdparams) | [config](./hrnet/hrnet_w32_256x256_mpii.yml) |
+
+场景模型
+| 模型 | 方案 | 输入尺寸 | 精度 | 预测速度 |模型权重 | 部署模型 | 说明|
+| :---- | ---|----- | :--------: | :--------: | :------------: |:------------: |:-------------------: |
+| HRNet-w32 + DarkPose | Top-Down|256x192 | AP: 87.1 (业务数据集)| 单人2.9ms |[下载链接](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.pdparams) |[下载链接](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) | 针对摔倒场景特别优化，该模型应用于[PP-Human](../../deploy/pipeline/README.md) |
+
+
+我们同时推出了基于LiteHRNet（Top-Down）针对移动端设备优化的实时关键点检测模型[PP-TinyPose](./tiny_pose/README.md), 欢迎体验。
-我们同时推出了针对移动端设备优化的实时关键点检测模型[PP-TinyPose](./tiny_pose/README.md), 欢迎体验。
## 快速开始
### 1、环境安装
- 请参考PaddleDetection [安装文档](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/INSTALL_cn.md)正确安装PaddlePaddle和PaddleDetection即可。
+ 请参考PaddleDetection [安装文档](../../docs/tutorials/INSTALL_cn.md)正确安装PaddlePaddle和PaddleDetection即可。
### 2、数据准备
- 目前KeyPoint模型支持[COCO](https://cocodataset.org/#keypoints-2017)数据集和[MPII](http://human-pose.mpi-inf.mpg.de/#overview)数据集，数据集的准备方式请参考[关键点数据准备](../../docs/tutorials/PrepareKeypointDataSet_cn.md)。
+ 目前KeyPoint模型支持[COCO](https://cocodataset.org/#keypoints-2017)数据集和[MPII](http://human-pose.mpi-inf.mpg.de/#overview)数据集，数据集的准备方式请参考[关键点数据准备](../../docs/tutorials/data/PrepareKeypointDataSet.md)。
关于config配置文件内容说明请参考[关键点配置文件说明](../../docs/tutorials/KeyPointConfigGuide_cn.md)。
-
- - 请注意，Top-Down方案使用检测框测试时，需要通过检测模型生成bbox.json文件。COCO val2017的检测结果可以参考[Detector having human AP of 56.4 on COCO val2017 dataset](https://paddledet.bj.bcebos.com/data/bbox.json)，下载后放在根目录（PaddleDetection）下，然后修改config配置文件中`use_gt_bbox: False`后生效。然后正常执行测试命令即可。
-
+- 请注意，Top-Down方案使用检测框测试时，需要通过检测模型生成bbox.json文件。COCO val2017的检测结果可以参考[Detector having human AP of 56.4 on COCO val2017 dataset](https://paddledet.bj.bcebos.com/data/bbox.json)，下载后放在根目录（PaddleDetection）下，然后修改config配置文件中`use_gt_bbox: False`后生效。然后正常执行测试命令即可。
### 3、训练与测试
- **单卡训练：**
+#### 单卡训练
```shell
#COCO DataSet
@@ -70,7 +114,7 @@ CUDA_VISIBLE_DEVICES=0 python3 tools/train.py -c configs/keypoint/higherhrnet/hi
CUDA_VISIBLE_DEVICES=0 python3 tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x256_mpii.yml
```
- **多卡训练：**
+#### 多卡训练
```shell
#COCO DataSet
@@ -80,7 +124,7 @@ CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m paddle.distributed.launch tools/train.py
CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m paddle.distributed.launch tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x256_mpii.yml
```
- **模型评估：**
+#### 模型评估
```shell
#COCO DataSet
@@ -93,7 +137,7 @@ CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/hrnet/hrnet_w32
CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml --save_prediction_only
```
- **模型预测：**
+#### 模型预测
注意：top-down模型只支持单人截图预测，如需使用多人图，请使用[联合部署推理]方式。或者使用bottom-up模型。
@@ -101,22 +145,32 @@ CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/higherhrnet/hig
CUDA_VISIBLE_DEVICES=0 python3 tools/infer.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml -o weights=./output/higherhrnet_hrnet_w32_512/model_final.pdparams --infer_dir=../images/ --draw_threshold=0.5 --save_txt=True
```
- **部署预测：**
+#### 模型部署
+
+##### Top-Down模型联合部署
+
+```shell
+#导出检测模型
+python tools/export_model.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams
+
+#导出关键点模型
+python tools/export_model.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml -o weights=https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams
+
+#detector 检测 + keypoint top-down模型联合部署（联合推理只支持top-down方式）
+python deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/ppyolo_r50vd_dcn_2x_coco/ --keypoint_model_dir=output_inference/hrnet_w32_384x288/ --video_file=../video/xxx.mp4 --device=gpu
+```
+
+##### Bottom-Up模型独立部署
```shell
#导出模型
python tools/export_model.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml -o weights=output/higherhrnet_hrnet_w32_512/model_final.pdparams
#部署推理
-#keypoint top-down/bottom-up 单独推理，该模式下top-down模型只支持单人截图预测。
python deploy/python/keypoint_infer.py --model_dir=output_inference/higherhrnet_hrnet_w32_512/ --image_file=./demo/000000014439_640x640.jpg --device=gpu --threshold=0.5
-python deploy/python/keypoint_infer.py --model_dir=output_inference/hrnet_w32_384x288/ --image_file=./demo/hrnet_demo.jpg --device=gpu --threshold=0.5
-
-#detector 检测 + keypoint top-down模型联合部署（联合推理只支持top-down方式）
-python deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/ppyolo_r50vd_dcn_2x_coco/ --keypoint_model_dir=output_inference/hrnet_w32_384x288/ --video_file=../video/xxx.mp4 --device=gpu
```
- **与多目标跟踪模型FairMOT联合部署预测：**
+##### 与多目标跟踪模型FairMOT联合部署
```shell
#导出FairMOT跟踪模型
@@ -125,18 +179,73 @@ python tools/export_model.py -c configs/mot/fairmot/fairmot_dla34_30e_1088x608.y
#用导出的跟踪和关键点模型Python联合预测
python deploy/python/mot_keypoint_unite_infer.py --mot_model_dir=output_inference/fairmot_dla34_30e_1088x608/ --keypoint_model_dir=output_inference/higherhrnet_hrnet_w32_512/ --video_file={your video name}.mp4 --device=GPU
```
+
**注意:**
- 跟踪模型导出教程请参考`configs/mot/README.md`。
+ 跟踪模型导出教程请参考[文档](../mot/README.md)。
+
+### 完整部署教程及Demo
+
+
+ 我们提供了PaddleInference(服务器端)、PaddleLite(移动端)、第三方部署(MNN、OpenVino)支持。无需依赖训练代码，deploy文件夹下相应文件夹提供独立完整部署代码。 详见 [部署文档](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/deploy/README.md)介绍。
+
+## 自定义数据训练
+
+我们以[tinypose_256x192](./tiny_pose/README.md)为例来说明对于自定义数据如何修改：
+
+#### 1、配置文件[tinypose_256x192.yml](../../configs/keypoint/tiny_pose/tinypose_256x192.yml)
+
+基本的修改内容及其含义如下：
+
+```
+num_joints: &num_joints 17 #自定义数据的关键点数量
+train_height: &train_height 256 #训练图片尺寸-高度h
+train_width: &train_width 192 #训练图片尺寸-宽度w
+hmsize: &hmsize [48, 64] #对应训练尺寸的输出尺寸，这里是输入[w,h]的1/4
+flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]] #关键点定义中左右对称的关键点，用于flip增强。若没有对称结构在 TrainReader 的 RandomFlipHalfBodyTransform 一栏中 flip_pairs 后面加一行 "flip: False"（注意缩紧对齐）
+num_joints_half_body: 8 #半身关键点数量，用于半身增强
+prob_half_body: 0.3 #半身增强实现概率，若不需要则修改为0
+upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] #上半身对应关键点id，用于半身增强中获取上半身对应的关键点。
+```
+
+上述是自定义数据时所需要的修改部分，完整的配置及含义说明可参考文件：[关键点配置文件说明](../../docs/tutorials/KeyPointConfigGuide_cn.md)。
+
+#### 2、其他代码修改（影响测试、可视化）
+- keypoint_utils.py中的sigmas = np.array([.26, .25, .25, .35, .35, .79, .79, .72, .72, .62, .62, 1.07, 1.07,.87, .87, .89, .89]) / 10.0，表示每个关键点的确定范围方差，根据实际关键点可信区域设置，区域精确的一般0.25-0.5，例如眼睛。区域范围大的一般0.5-1.0，例如肩膀。若不确定建议0.75。
+- visualizer.py中的draw_pose函数中的EDGES，表示可视化时关键点之间的连接线关系。
+- pycocotools工具中的sigmas，同第一个keypoint_utils.py中的设置。用于coco指标评估时计算。
+
+#### 3、数据准备注意
+- 训练数据请按coco数据格式处理。需要包括关键点[Nx3]、检测框[N]标注。
+- 请注意area>0，area=0时数据在训练时会被过滤掉。此外，由于COCO的评估机制，area较小的数据在评估时也会被过滤掉，我们建议在自定义数据时取`area = bbox_w * bbox_h`。
+
+如有遗漏，欢迎反馈
-### 4、模型单独部署
- 我们提供了PaddleInference(服务器端)、PaddleLite(移动端)、第三方部署(MNN、OpenVino)支持。无需依赖训练代码，deploy文件夹下相应文件夹提供独立完整部署代码。
-详见 [部署文档](../../deploy/README.md)介绍。
+## 关键点稳定策略（仅适用于视频数据）
+使用关键点算法处理视频数据时，由于预测针对单帧图像进行，在视频结果上往往会有抖动的现象。在一些依靠精细化坐标的应用场景（例如健身计数、基于关键点的虚拟渲染等）上容易造成误检或体验不佳的问题。针对这个问题，在PaddleDetection关键点视频推理中加入了[OneEuro滤波器](http://www.lifl.fr/~casiez/publications/CHI2012-casiez.pdf)和EMA两种关键点稳定方式。实现将当前关键点坐标结果和历史关键点坐标结果结合计算，使得输出的点坐标更加稳定平滑。该功能同时支持在Python及C++推理中一键开启使用。
-## Benchmark
-我们给出了不同运行环境下的测试结果，供您在选用模型时参考。详细数据请见[Keypoint Inference Benchmark](./KeypointBenchmark.md)。
+```bash
+# 使用Python推理
+python deploy/python/det_keypoint_unite_infer.py \
+ --det_model_dir output_inference/picodet_s_320 \
+ --keypoint_model_dir output_inference/tinypose_256x192 \
+ --video_file test_video.mp4 --device gpu --smooth True
+
+# 使用CPP推理
+./deploy/cpp/build/main --det_model_dir output_inference/picodet_s_320 \
+ --keypoint_model_dir output_inference/tinypose_256x192 \
+ --video_file test_video.mp4 --device gpu --smooth True
+```
+效果如下：
+
+
+
+## BenchMark
+
+我们给出了不同运行环境下的测试结果，供您在选用模型时参考。详细数据请见[Keypoint Inference Benchmark](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/configs/keypoint/KeypointBenchmark.md)。
## 引用
+
```
@inproceedings{cheng2020bottom,
title={HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation},
diff --git a/configs/keypoint/README_en.md b/configs/keypoint/README_en.md
index 05e77f66819c26a38a54746eeb9e569f4945b442..f6f049824eb5aa185b767f37648bed429196d913 100644
--- a/configs/keypoint/README_en.md
+++ b/configs/keypoint/README_en.md
@@ -2,19 +2,63 @@
# KeyPoint Detection Models
-
+## Content
+
+- [Introduction](#introduction)
+- [Model Recommendation](#model-recommendation)
+- [Model Zoo](#model-zoo)
+- [Getting Start](#getting-start)
+ - [Environmental Installation](#1environmental-installation)
+ - [Dataset Preparation](#2dataset-preparation)
+ - [Training and Testing](#3training-and-testing)
+ - [Training on single GPU](#training-on-single-gpu)
+ - [Training on multiple GPU](#training-on-multiple-gpu)
+ - [Evaluation](#evaluation)
+ - [Inference](#inference)
+ - [Deploy Inference](#deploy-inference)
+ - [Deployment for Top-Down models](#deployment-for-top-down-models)
+ - [Deployment for Bottom-Up models](#deployment-for-bottom-up-models)
+ - [Joint Inference with Multi-Object Tracking Model FairMOT](#joint-inference-with-multi-object-tracking-model-fairmot)
+ - [Complete Deploy Instruction and Demo](#complete-deploy-instruction-and-demo)
+- [Train with custom data](#train-with-custom-data)
+- [BenchMark](#benchmark)
## Introduction
-- The keypoint detection part in PaddleDetection follows the state-of-the-art algorithm closely, including Top-Down and BottomUp methods, which can meet the different needs of users.
+The keypoint detection part in PaddleDetection follows the state-of-the-art algorithm closely, including Top-Down and Bottom-Up methods, which can satisfy the different needs of users. Top-Down detects the object first and then detects the specific keypoint. Top-Down models will be more accurate, but slower as the number of objects increases. Differently, Bottom-Up detects the point first and then group or connect those points to form several instances of human pose. The speed of Bottom-Up is fixed, it won't slow down as the number of objects increases, but it will be less accurate.
+
+At the same time, PaddleDetection provides a self-developed real-time keypoint detection model [PP-TinyPose](./tiny_pose/README_en.md) optimized for mobile devices.
+## Model Recommendation
+
+### Mobile Terminal
+
+
+
+
+| Detection Model | Keypoint Model | Input Size | Accuracy of COCO | Average Inference Time (FP16) | Params (M) | Flops (G) | Model Weight | Paddle-Lite Inference Model（FP16) |
+| :----------------------------------------------------------- | :------------------------------------ | :-------------------------------------: | :--------------------------------------: | :-----------------------------------: | :--------------------------------: | :--------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
+| [PicoDet-S-Pedestrian](../picodet/legacy_model/application/pedestrian_detection/picodet_s_192_pedestrian.yml) | [PP-TinyPose](./tiny_pose/tinypose_128x96.yml) | Detection：192x192
Keypoint：128x96 | Detection mAP：29.0
Keypoint AP：58.1 | Detection：2.37ms
Keypoint：3.27ms | Detection：1.18
Keypoint：1.36 | Detection：0.35
Keypoint：0.08 | [Detection](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian.pdparams)
[Keypoint](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96.pdparams) | [Detection](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_192_pedestrian_fp16.nb)
[Keypoint](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_128x96_fp16.nb) |
+| [PicoDet-S-Pedestrian](../picodet/legacy_model/application/pedestrian_detection/picodet_s_320_pedestrian.yml) | [PP-TinyPose](./tiny_pose/tinypose_256x192.yml) | Detection：320x320
Keypoint：256x192 | Detection mAP：38.5
Keypoint AP：68.8 | Detection：6.30ms
Keypoint：8.33ms | Detection：1.18
Keypoint：1.36 | Detection：0.97
Keypoint：0.32 | [Detection](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian.pdparams)
[Keypoint](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192.pdparams) | [Detection](https://bj.bcebos.com/v1/paddledet/models/keypoint/picodet_s_320_pedestrian_fp16.nb)
[Keypoint](https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_256x192_fp16.nb) |
+
+
+*Specific documents of PP-TinyPose, please refer to [Document](./tiny_pose/README.md)。
+
+### Terminal Server
+
+| Detection Model | Keypoint Model | Input Size | Accuracy of COCO | Params (M) | Flops (G) | Model Weight |
+| :----------------------------------------------------------- | :----------------------------------------- | :-------------------------------------: | :--------------------------------------: | :-----------------------------: | :-----------------------------: | :----------------------------------------------------------: |
+| [PP-YOLOv2](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [HRNet-w32](./hrnet/hrnet_w32_384x288.yml) | Detection：640x640
Keypoint：384x288 | Detection mAP：49.5
Keypoint AP：77.8 | Detection：54.6
Keypoint：28.6 | Detection：115.8
Keypoint：17.3 | [Detection](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams)
[Keypoint](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams) |
+| [PP-YOLOv2](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.3/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml) | [HRNet-w32](./hrnet/hrnet_w32_256x192.yml) | Detection：640x640
Keypoint：256x192 | Detection mAP：49.5
Keypoint AP：76.9 | Detection：54.6
Keypoint：28.6 | Detection：115.8
Keypoint：7.68 | [Detection](https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams)
[Keypoint](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_384x288.pdparams) |
+
+
+## Model Zoo
-#### Model Zoo
COCO Dataset
| Model | Input Size | AP(coco val) | Model Download | Config File |
| :---------------- | -------- | :----------: | :----------------------------------------------------------: | ----------------------------------------------------------- |
@@ -31,7 +75,6 @@ COCO Dataset
| LiteHRNet-30 | 256x192 | 69.4 | [lite_hrnet_30_256x192_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_30_256x192_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_30_256x192_coco.yml) |
| LiteHRNet-30 | 384x288 | 72.5 | [lite_hrnet_30_384x288_coco.pdparams](https://bj.bcebos.com/v1/paddledet/models/keypoint/lite_hrnet_30_384x288_coco.pdparams) | [config](./lite_hrnet/lite_hrnet_30_384x288_coco.yml) |
-
Note：The AP results of Top-Down models are based on bounding boxes in GroundTruth.
MPII Dataset
@@ -40,25 +83,31 @@ MPII Dataset
| HRNet-w32 | 256x256 | 90.6 | 38.5 | [hrnet_w32_256x256_mpii.pdparams](https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x256_mpii.pdparams) | [config](./hrnet/hrnet_w32_256x256_mpii.yml) |
+Model for Scenes
+| Model | Strategy | Input Size | Precision | Inference Speed |Model Weights | Model Inference and Deployment | description|
+| :---- | ---|----- | :--------: | :-------: |:------------: |:------------: |:-------------------: |
+| HRNet-w32 + DarkPose | Top-Down|256x192 | AP: 87.1 (on internal dataset)| 2.9ms per person |[Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.pdparams) |[Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) | Especially optimized for fall scenarios, the model is applied to [PP-Human](../../deploy/pipeline/README.md) |
+
+
We also release [PP-TinyPose](./tiny_pose/README_en.md), a real-time keypoint detection model optimized for mobile devices. Welcome to experience.
## Getting Start
-### 1. Environmental Installation
+### 1.Environmental Installation
- Please refer to [PaddleDetection Installation Guild](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/INSTALL.md) to install PaddlePaddle and PaddleDetection correctly.
+ Please refer to [PaddleDetection Installation Guide](../../docs/tutorials/INSTALL.md) to install PaddlePaddle and PaddleDetection correctly.
-### 2. Dataset Preparation
+### 2.Dataset Preparation
- Currently, KeyPoint Detection Models support [COCO](https://cocodataset.org/#keypoints-2017) and [MPII](http://human-pose.mpi-inf.mpg.de/#overview). Please refer to [Keypoint Dataset Preparation](../../docs/tutorials/PrepareKeypointDataSet_en.md) to prepare dataset.
+ Currently, KeyPoint Detection Models support [COCO](https://cocodataset.org/#keypoints-2017) and [MPII](http://human-pose.mpi-inf.mpg.de/#overview). Please refer to [Keypoint Dataset Preparation](../../docs/tutorials/data/PrepareKeypointDataSet_en.md) to prepare dataset.
About the description for config files, please refer to [Keypoint Config Guild](../../docs/tutorials/KeyPointConfigGuide_en.md).
- - Note that, when testing by detected bounding boxes in Top-Down method, We should get `bbox.json` by a detection model. You can download the detected results for COCO val2017 [(Detector having human AP of 56.4 on COCO val2017 dataset)](https://paddledet.bj.bcebos.com/data/bbox.json) directly, put it at the root path (`PaddleDetection/`), and set `use_gt_bbox: False` in config file.
+- Note that, when testing by detected bounding boxes in Top-Down method, We should get `bbox.json` by a detection model. You can download the detected results for COCO val2017 [(Detector having human AP of 56.4 on COCO val2017 dataset)](https://paddledet.bj.bcebos.com/data/bbox.json) directly, put it at the root path (`PaddleDetection/`), and set `use_gt_bbox: False` in config file.
-### 3、Training and Testing
+### 3.Training and Testing
- **Training on single gpu:**
+#### Training on single GPU
```shell
#COCO DataSet
@@ -68,7 +117,7 @@ CUDA_VISIBLE_DEVICES=0 python3 tools/train.py -c configs/keypoint/higherhrnet/hi
CUDA_VISIBLE_DEVICES=0 python3 tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x256_mpii.yml
```
- **Training on multiple gpu:**
+#### Training on multiple GPU
```shell
#COCO DataSet
@@ -78,7 +127,7 @@ CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m paddle.distributed.launch tools/train.py
CUDA_VISIBLE_DEVICES=0,1,2,3 python3 -m paddle.distributed.launch tools/train.py -c configs/keypoint/hrnet/hrnet_w32_256x256_mpii.yml
```
- **Evaluation**
+#### Evaluation
```shell
#COCO DataSet
@@ -91,7 +140,7 @@ CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/hrnet/hrnet_w32
CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml --save_prediction_only
```
- **Inference**
+#### Inference
Note：Top-down models only support inference for a cropped image with single person. If you want to do inference on image with several people, please see "joint inference by detection and keypoint". Or you can choose a Bottom-up model.
@@ -99,22 +148,34 @@ CUDA_VISIBLE_DEVICES=0 python3 tools/eval.py -c configs/keypoint/higherhrnet/hig
CUDA_VISIBLE_DEVICES=0 python3 tools/infer.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml -o weights=./output/higherhrnet_hrnet_w32_512/model_final.pdparams --infer_dir=../images/ --draw_threshold=0.5 --save_txt=True
```
- **Deploy Inference**
+#### Deploy Inference
+
+##### Deployment for Top-Down models
```shell
-#export models
-python tools/export_model.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml -o weights=output/higherhrnet_hrnet_w32_512/model_final.pdparams
+#Export Detection Model
+python tools/export_model.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/ppyolov2_r50vd_dcn_365e_coco.pdparams
-#deploy inference
-#keypoint inference for a single model of top-down/bottom-up method. In this mode, top-down model only support inference for a cropped image with single person.
-python deploy/python/keypoint_infer.py --model_dir=output_inference/higherhrnet_hrnet_w32_512/ --image_file=./demo/000000014439_640x640.jpg --device=gpu --threshold=0.5
-python deploy/python/keypoint_infer.py --model_dir=output_inference/hrnet_w32_384x288/ --image_file=./demo/hrnet_demo.jpg --device=gpu --threshold=0.5
-#joint inference by detection and keypoint for top-down models.
+#Export Keypoint Model
+python tools/export_model.py -c configs/keypoint/hrnet/hrnet_w32_256x192.yml -o weights=https://paddledet.bj.bcebos.com/models/keypoint/hrnet_w32_256x192.pdparams
+
+#Deployment for detector and keypoint, which is only for Top-Down models
python deploy/python/det_keypoint_unite_infer.py --det_model_dir=output_inference/ppyolo_r50vd_dcn_2x_coco/ --keypoint_model_dir=output_inference/hrnet_w32_384x288/ --video_file=../video/xxx.mp4 --device=gpu
```
- **joint inference with Multi-Object Tracking model FairMOT**
+##### Deployment for Bottom-Up models
+
+```shell
+#Export model
+python tools/export_model.py -c configs/keypoint/higherhrnet/higherhrnet_hrnet_w32_512.yml -o weights=output/higherhrnet_hrnet_w32_512/model_final.pdparams
+
+
+#Keypoint independent deployment, which is only for bottom-up models
+python deploy/python/keypoint_infer.py --model_dir=output_inference/higherhrnet_hrnet_w32_512/ --image_file=./demo/000000014439_640x640.jpg --device=gpu --threshold=0.5
+```
+
+##### Joint Inference with Multi-Object Tracking Model FairMOT
```shell
#export FairMOT model
@@ -123,17 +184,51 @@ python tools/export_model.py -c configs/mot/fairmot/fairmot_dla34_30e_1088x608.y
#joint inference with Multi-Object Tracking model FairMOT
python deploy/python/mot_keypoint_unite_infer.py --mot_model_dir=output_inference/fairmot_dla34_30e_1088x608/ --keypoint_model_dir=output_inference/higherhrnet_hrnet_w32_512/ --video_file={your video name}.mp4 --device=GPU
```
+
**Note:**
To export MOT model, please refer to [Here](../../configs/mot/README_en.md).
-### 4、Deploy standalone
+### Complete Deploy Instruction and Demo
+
+ We provide standalone deploy of PaddleInference(Server-GPU)、PaddleLite(mobile、ARM)、Third-Engine(MNN、OpenVino), which is independent of training codes。For detail, please click [Deploy-docs](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/deploy/README_en.md)。
+
+## Train with custom data
+
+We take an example of [tinypose_256x192](./tiny_pose/README_en.md) to show how to train with custom data.
- We provide standalone deploy of PaddleInference(Server-GPU)、PaddleLite(mobile、ARM)、Third-Engine(MNN、OpenVino), which is independent of training codes。For detail, please click [Deploy-docs](../../deploy/README_en.md)。
+#### 1、For configs [tinypose_256x192.yml](../../configs/keypoint/tiny_pose/tinypose_256x192.yml)
-## Benchmark
-We provide benchmarks in different runtime environments for your reference when choosing models. See [Keypoint Inference Benchmark](./KeypointBenchmark.md) for details.
+you may need to modity these for your job：
+
+```
+num_joints: &num_joints 17 #the number of joints in your job
+train_height: &train_height 256 #the height of model input
+train_width: &train_width 192 #the width of model input
+hmsize: &hmsize [48, 64] #the shape of model output，usually 1/4 of [w,h]
+flip_perm: &flip_perm [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]] #the correspondence between left and right keypoint id，used for flip transform。You can add an line(by "flip: False") behind of flip_pairs in RandomFlipHalfBodyTransform of TrainReader if you don't need it
+num_joints_half_body: 8 #The joint numbers of half body, used for half_body transform
+prob_half_body: 0.3 #The probility of half_body transform, set to 0 if you don't need it
+upper_body_ids: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] #The joint ids of half(upper) body, used to get the upper joints in half_body transform
+```
+
+For more configs, please refer to [KeyPointConfigGuide](../../docs/tutorials/KeyPointConfigGuide_en.md)。
+
+#### 2、Others(used for test and visualization)
+- In keypoint_utils.py, please set: "sigmas = np.array([.26, .25, .25, .35, .35, .79, .79, .72, .72, .62, .62, 1.07, 1.07,.87, .87, .89, .89]) / 10.0", the value indicate the variance of a joint locations，normally 0.25-0.5 means the location is highly accuracy，for example: eyes。0.5-1.0 means the location is not sure so much，for example: shoulder。0.75 is recommand if you not sure。
+- In visualizer.py, please set "EDGES" in draw_pose function，this indicate the line to show between joints for visualization。
+- In pycocotools you installed, please set "sigmas"，it is the same as that in keypoint_utils.py, but used for coco evaluation。
+
+#### 3、Note for data preparation
+- The data should has the same format as Coco data, and the keypoints(Nx3) and bbox(N) should be annotated.
+- please set "area">0 in annotations files otherwise it will be skiped while training. Moreover, due to the evaluation mechanism of COCO, the data with small area may also be filtered out during evaluation. We recommend to set `area = bbox_w * bbox_h` when customizing your dataset.
+
+
+## BenchMark
+
+We provide benchmarks in different runtime environments for your reference when choosing models. See [Keypoint Inference Benchmark](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/configs/keypoint/KeypointBenchmark.md) for details.
## Reference
+
```
@inproceedings{cheng2020bottom,
title={HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation},
diff --git a/configs/keypoint/tiny_pose/README.md b/configs/keypoint/tiny_pose/README.md
index 6d9c5be02d4eccfff00abc611da96296fab2223c..809b0ab10f30f74969e5f4773ddad1e8add2c8b6 100644
--- a/configs/keypoint/tiny_pose/README.md
+++ b/configs/keypoint/tiny_pose/README.md
@@ -7,12 +7,20 @@
图片来源：COCO2017开源数据集
+ 
+
+
+ 
@@ -43,20 +49,46 @@ PP-Tracking 提供了简洁的GUI可视化界面，教程请参考[PP-Tracking
视频来源：VisDrone和BDD100K公开数据集
+
+
+| 模型 | 数据集 | SLICE_SIZE | OVERLAP_RATIO | 类别数 | mAPval
+
+
+
+
+ 
+
+ 
+
+ 
+
+
+
+