/multi-human-pose

Training code for the SPPE and SSTN of our RMPE framework

Primary LanguageLua

RMPE: Regional Multi-person Pose Estimation(SPPE+SSTN part)

This repository includes Torch code for training our SPPE+SSTN model presented in our arXiv paper.

Contents

  1. Installation
  2. Preparation
  3. Evaluate
  4. Train
  5. Acknowledgements

Installation

To run this code, the following must be installed:

To install STN module, do

git clone https://github.com/Fang-Haoshu/stnbhwd.git
cd stnbhwd && luarocks make stnbhwd-scm-1.rockspec

After all the installation is done, get the code. We will call the directory that you cloned this repo into $SPPE_ROOT

git clone https://github.com/Fang-Haoshu/multi-human-pose.git
cd multi-human-pose

Preparation

For evaluation only

  1. Download pre-trained SPPE+SSTN model(Google drive|Baidu cloud). By default, we assume the models are stored in $SPPE_ROOT/predict/model/.

For training

  1. Download stacked hourglass networks. By default, we assume the model is stored in $SPPE_ROOT/train/.

  2. Download MPII images and COCO14 training set. By default, we assume the images are stored in /data/MPII_COCO14/images/.

Evaluate

You will need to generate bounding box first. Here we have already generated the bounding box in $SPPE_ROOT/predict/annot/mpii-test0.09/. To generate it yourself, please follow the guidlines in the main repo.

cd $SPPE_ROOT/predict
# make a soft link to the images then test
ln -s /data/MPII_COCO14/images/ data/images
# get the predicted results
th main.lua predict-test

You can also use the following command to visualize the single person pose results.

  qlua main.lua demo

Train

We finetune our model based on the pre-trained stacked-hourglass model.

cd $SPPE_ROOT/train/src
ln -s /data/MPII_COCO14/images ../data/mpii/images
# First finetune the model using DPG
th main.lua -expID finetune -addDPG
th main.lua -expID finetune -addDPG -continue -LR 0.5e-4 -nEpochs 10
# Then add parallel SPPE and SSTN on the finetuned model
# It should reach a final mAP of 80.*
th main.lua -expID final_model -loadModel '../exp/mpii/finetune/final_model.t7' -LR 0.5e-4 -addParallelSPPE -addSTN -addDPG
th main.lua -expID final_model -continue -Npochs 8 -LR 0.1e-4 -addParallelSPPE -addSTN -addDPG

Acknowledgements

Thanks to Wei Liu, Alejandro Newell, Pfister, T., Kaichun Mo, Maxime Oquab for contributing their codes. Thanks to the authors of Caffe and Torch7!