DeepCV/multi-human-pose

Training code for the SPPE and SSTN of our RMPE framework

RMPE: Regional Multi-person Pose Estimation(SPPE+SSTN part)

This repository includes Torch code for training our SPPE+SSTN model presented in our arXiv paper.

Contents

1. Installation
2. Preparation
3. Evaluate
4. Train
5. Acknowledgements

Installation

To run this code, the following must be installed:

• Torch7
• hdf5 (and the torch-hdf5 package)
• cudnn
• qlua (for displaying results)
• STN module(use this stable repo)

To install STN module, do

git clone https://github.com/Fang-Haoshu/stnbhwd.git
cd stnbhwd && luarocks make stnbhwd-scm-1.rockspec

After all the installation is done, get the code. We will call the directory that you cloned this repo into $SPPE_ROOT

git clone https://github.com/Fang-Haoshu/multi-human-pose.git
cd multi-human-pose

Preparation

For evaluation only

1. Download pre-trained SPPE+SSTN model(Google drive|Baidu cloud). By default, we assume the models are stored in $SPPE_ROOT/predict/model/.

For training

1. Download stacked hourglass networks. By default, we assume the model is stored in $SPPE_ROOT/train/.

2. Download MPII images and COCO14 training set. By default, we assume the images are stored in /data/MPII_COCO14/images/.

Evaluate

You will need to generate bounding box first. Here we have already generated the bounding box in $SPPE_ROOT/predict/annot/mpii-test0.09/. To generate it yourself, please follow the guidlines in the main repo.

cd $SPPE_ROOT/predict
# make a soft link to the images then test
ln -s /data/MPII_COCO14/images/ data/images
# get the predicted results
th main.lua predict-test

You can also use the following command to visualize the single person pose results.

  qlua main.lua demo

Train

We finetune our model based on the pre-trained stacked-hourglass model.

cd $SPPE_ROOT/train/src
ln -s /data/MPII_COCO14/images ../data/mpii/images
# First finetune the model using DPG
th main.lua -expID finetune -addDPG
th main.lua -expID finetune -addDPG -continue -LR 0.5e-4 -nEpochs 10
# Then add parallel SPPE and SSTN on the finetuned model
# It should reach a final mAP of 80.*
th main.lua -expID final_model -loadModel '../exp/mpii/finetune/final_model.t7' -LR 0.5e-4 -addParallelSPPE -addSTN -addDPG
th main.lua -expID final_model -continue -Npochs 8 -LR 0.1e-4 -addParallelSPPE -addSTN -addDPG

Acknowledgements

Thanks to Wei Liu, Alejandro Newell, Pfister, T., Kaichun Mo, Maxime Oquab for contributing their codes. Thanks to the authors of Caffe and Torch7!