Object-Occluded Human Shape and Pose Estimation from a Single Color Image (CVPR2020 Oral)

Introduction

This is the offical implentation of 'Object-Occluded Human Shape and Pose Estimation from a Single Color Image' (CVPR2020 Oral/TPAMI2022). [Project Page]

News!

• Feb 2023: We released the video version of 3DOH50K dataset link.
• Mar 2022: We add mesh fitting codes which fit SMPL model to the regressed mesh.
• Aug 2021: New model with ResNet-50 backbone. A heatmap branch is added to overlay regressed meshes to RGB images.

Environment

This code is based on Python 3.6, CUDA 10.0, cuDNN 7.6 on Windows10.

Getting Started

Clone the repo:

git clone https://gitee.com/seuvcl/CVPR2020-OOH.git

Install the requirements using conda:

conda create -n occlusion python=3.6
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=10.1 -c pytorch
pip install -r requirements.txt

Running the Demo

1. Download the trained model from here (extraction code [jhwp]) and put it in trained_model folder.

2. Download the official SMPL neutral model. Rename the model to SMPL_NEUTRAL.pkl and put it in data folder.

3. To test on your own image, you can edit the cfg_files\demo.yaml and run:

python demo.py --config cfg_files\demo.yaml

If you want to fit the SMPL model to the regressed mesh, you can set the fitting=True in cfg_files\demo.yaml and then run:

python demo.py --config cfg_files\demo.yaml

Dataset

We provide the 3DOH50K dataset (extraction code [hb1d]), which is the first real 3D human dataset for the problem of human reconstruction and pose estimation in occlusion scenarios. Visualizing 3DOH50K:

python utils/visualize.py --base_dir /PATH/TO/THE/3DOH50K

We also provide the full video dataset at here.

Citation

@article{huang2022object,
  title={Object-Occluded Human Shape and Pose Estimation with Probabilistic Latent Consistency},
  author={Huang, Buzhen and Zhang, Tianshu and Wang, Yangang},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2022},
  publisher={IEEE}
}

@inproceedings{ooh20,
  title = {Object-Occluded Human Shape and Pose Estimation from a Single Color Image},
  author = {Tianshu, Zhang and Buzhen, Huang and Yangang, Wang},
  booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
  year = {2020}
}

References

PyTorch implementation of SMPL model is from CalciferZh.