yolov5-smoking-detect

A C++ implementation of Yolov5 to detect peaple smoking running in Jetson Xavier nx and Jetson nano.In Jetson Xavier Nx, it can achieve 33 FPS.

You can see video play in BILIBILI, or YOUTUBE.

If you want to try to train your own model, you can see yolov5-smoke-detection-python. Follow the readme to get your own model.

Requirement

1. Jetson nano or Jetson Xavier nx
2. Jetpack 4.5.1
3. python3 with default(jetson nano or jetson xavier nx has default python3 with tensorrt 7.1.3.0 )
4. tensorrt 7.1.3.0
5. torch 1.8.0
6. torchvision 0.9.0
7. torch2trt 0.3.0
8. onnx 1.4.1
9. opencv-python 4.5.3.56
10. protobuf 3.17.3
11. scipy 1.5.4

if you have problem in this project, you can see this CSDN artical.

Achieve and Experiment

• Int8.
• yolov5-s
• yolov5-m

Comming soon

• Faster and use less memory.

Speed

Whole process time from read image to finish process (include every img preprocess and postprocess). And all results can get in Jetson Xavier nx. For python model and code, you can find them in this yolov5-smoke-detection-python

Build and Run

git clone https://github.com/RichardoMrMu/yolov5-smoking-detect
cd yolov5-smoking-detect
mkdir build 
cmake ..
make 

if you meet some errors in cmake and make, please see this artical or see Attention.

Model

You need two model, one is yolov5 model, for detection, generating from tensorrtx.

Generate yolov5 model

For yolov5 detection model, I choose yolov5s, and choose yolov5s.pt->yolov5s.wts->yolov5s.engine Note that, used models can get from yolov5 and use this yolov5-smoke-detection-python to get your model. You can also see tensorrtx official readme

1. Get yolov5 repository

Note that, here uses the official pertained model.And I use yolov5-5, v5.0. So if you train your own model, please be sure your yolov5 code is v5.0.

git clone -b v5.0 https://github.com/ultralytics/yolov5.git
cd yolov5
mkdir weights
cd weights
// download https://github.com/ultralytics/yolov5/releases/download/v5.0/yolov5s.pt
wget https://github.com/ultralytics/yolov5/releases/download/v5.0/yolov5s.pt

2. Get tensorrtx.

git clone https://github.com/wang-xinyu/tensorrtx

3. Get xxx.wst model

cp tensorrtx/gen_wts.py yolov5/
cd yolov5 
python3 gen_wts.py -w ./weights/yolov5s.pt -o ./weights/yolov5s.wts
// a file 'yolov5s.wts' will be generated.

You can get yolov5s.wts model in yolov5/weights/

4. Build tensorrtx/yolov5 and get tensorrt engine

cd tensorrtx/yolov5
// update CLASS_NUM to 2 in yololayer.h 
// nc: 1  # number of classes
// names: ['smoke']  # class names

mkdir build
cd build
cp {ultralytics}/yolov5/yolov5s.wts {tensorrtx}/yolov5/build
cmake ..
make
// yolov5s
sudo ./yolov5 -s yolov5s.wts yolov5s.engine s
// test your engine file
sudo ./yolov5 -d yolov5s.engine ../samples

Then you get the yolov5s.engine, and you can put yolov5s.engine in My project. For example

cd {yolov5-smoking-detect}
mkdir resources
cp {tensorrtx}/yolov5/build/yolov5s.engine {yolov5-smoking-detect}/resources

You may face some problems in getting yolov5s.engine, you can upload your issue in github or csdn artical.

Run Your Custom Model

You may need train your own model and transfer your trained-model to tensorRT. So you can follow the following steps.

1. Train Custom Model You can follow the official wiki to train your own model on your dataset. For example, I choose yolov5-s to train my model.
2. Transfer Custom Model Just like tensorRT official guideline.When your follow Generate yolov5 model to get yolov5 and tensorrt rep, next step is to transfer your pytorch model to tensorrt. Before this, you need to change yololayer.h file 20,21 and 22 line(CLASS_NUM，INPUT_H,INPUT_W) to your own parameters.

// before 
static constexpr int CLASS_NUM = 80; // 20
static constexpr int INPUT_H = 640;  // 21  yolov5's input height and width must be divisible by 32.
static constexpr int INPUT_W = 640; // 22

// after 
// if your model is 2 classfication and image size is 416*416
static constexpr int CLASS_NUM = 2; // 20
static constexpr int INPUT_H = 416;  // 21  yolov5's input height and width must be divisible by 32.
static constexpr int INPUT_W = 416; // 22

cd {tensorrtx}/yolov5/
// update CLASS_NUM in yololayer.h if your model is trained on custom dataset

mkdir build
cd build
cp {ultralytics}/yolov5/yolov5s.wts {tensorrtx}/yolov5/build
cmake ..
make
sudo ./yolov5 -s [.wts] [.engine] [s/m/l/x/s6/m6/l6/x6 or c/c6 gd gw]  // serialize model to plan file
sudo ./yolov5 -d [.engine] [image folder]  // deserialize and run inference, the images in [image folder] will be processed.
// For example yolov5s
sudo ./yolov5 -s yolov5s.wts yolov5s.engine s
sudo ./yolov5 -d yolov5s.engine ../samples
// For example Custom model with depth_multiple=0.17, width_multiple=0.25 in yolov5.yaml
sudo ./yolov5 -s yolov5_custom.wts yolov5.engine c 0.17 0.25
sudo ./yolov5 -d yolov5.engine ../samples

In this way, you can get your own tensorrt yolov5 model. Enjoy it!

INT8 Quantization

It has some diffirence between float16 tensorrt engine file and int8. Just like tensorrtx readme, Int8 engine file needs calibration images.

For official yolov5 model , you need to downlowd coco_calid.zip from this google drive url or BAIDUYUN --- a9wh . And unzip to {project}/build/.

Then change yolov5.cpp's 10 line from USE_FLOAT16 to USE_INT8.And run this :

cmake ..
make 
// yolov5s
sudo ./yolov5 -s yolov5s.wts yolov5s-int8.engine s
// testyour engine file
sudo ./yolov5 -d yolov5s-int8.engine ../samples

Other Project

• yolov5-deepsort-tensorrt
• facial-emotion-recognition
• yolov5-fire-smoke-detect-python
• yolov5-helmet-detection
• yolov5-fire-smoke-detect
• gazecapture-web gaze estimation
• yolov5-helmet-detection-python
• yolov5-reflective-clothes-detect-python
• yolov5-reflective-clothes-detect
• yolov5-smoking-detect
• yolov5-mask-detect
• yolov5-mask-detection-python
• yolov5-smoke-detection-python
• deepsort-tensorrt