Convolutional Pose Machines

Shih-En Wei, Varun Ramakrishna, Takeo Kanade, Yaser Sheikh, "Convolutional Pose Machines", CVPR 2016.

This project is licensed under the terms of the GPL v2 license. By using the software, you are agreeing to the terms of the license agreement.

Contact: shihenw@cmu.edu.

Before Everything

Watch some videos.
Install Caffe. If you are interested in training this model on your own machines, consider using our version with a data layer performing online augmentation. Make sure you have done make matcaffe and make pycaffe.
Copy caffePath.cfg.example to caffePath.cfg and set your own path in it.

Testing

Python

This demo file shows how to detect multiple people's poses as we demonstrated in CVPR'16. For real-time performance, please read it for further explanation.

Matlab

Run testing/get_model.sh to retreive trained models from our web server.
1. CPM_demo.m: Put the testing image into sample_image then run it! You can select models (we provided 4) or other parameters in config.m. If you just want to try our best-scoring model, leave them default.
1. CPM_benchmark.m: Run the model on test benchmark and see the scores. Prediction files will be saved in testing/predicts.

Training

Run get_data.sh to get datasets including FLIC Dataset, LEEDS Sport Dataset and its extended training set, and MPII Dataset.
Run genJSON(<dataset_name>) to generate a json file in training/json/ folder. Dataset name can be MPI, LEEDS, or FLIC. The json files contain raw informations needed for training from each individual dataset.
Run python genLMDB.py to generate LMDBs for CPM data layer in our caffe. Change the main function to select dataset, and note that you can generate a LMDB with multiple datasets.
Run python genProto.py to get prototxt for caffe. Read further explanation for layer parameters.
Train with generated prototxts and collect caffemodels.

Citation

Please cite CPM in your publications if it helps your research:

@inproceedings{wei2016cpm,
    author = {Shih-En Wei and Varun Ramakrishna and Takeo Kanade and Yaser Sheikh},
    booktitle = {CVPR},
    title = {Convolutional pose machines},
    year = {2016}
}