feitiandemiaomi/tf-openpose

Openpose from CMU implemented using Tensorflow

tf-openpose

Openpose from CMU implemented using Tensorflow. It also provides several variants that have made some changes to the network structure for real-time processing on the CPU.

Original Repo(Caffe) : https://github.com/CMU-Perceptual-Computing-Lab/openpose

Features

• CMU's original network architecture and weights.

  • Transfer Original Weights to Tensorflow

  • Training Code with multi-gpus

• Post processing from network output.

  • Accurate Result

  • Fast Computation

• Faster network variants using mobilenet, lcnn architecture.

  • Depthwise Separable Convolution Version

  • Mobilenet Version

  • LCNN Version

• Demos

  • Realtime Webcam Demo

  • Image/Video File Demo

• ROS Support.

Install

Dependencies

You need dependencies below.

• python3

• tensorflow 1.3

• opencv3

• protobuf

• python3-tk

Install

$ git clone https://www.github.com/ildoonet/tf-openpose
$ cd tf-openpose
$ git lfs pull
$ pip3 install -r requirements.txt

Models

Inference Time

Dataset	Model	Description	Inference Time 1 core cpu
Coco	cmu	CMU's original version. Same network, same weights.	368x368 @ 10s 320x240 @ 3.65s
Coco	dsconv	Same architecture as the cmu version except for the depthwise separable convolution of mobilenet.	368x368 @ 1.1s 320x240 @ 0.44s
Coco	mobilenet	Feature extraction layers is replaced from VGG to Mobilenet from Google

• Test being processed. This will be updated soon.

Training

Dataset

Augmentation

CMU Perceptual Computing Lab has modified Caffe to provide data augmentation. See : https://github.com/CMU-Perceptual-Computing-Lab/caffe_train

I implemented the augmentation codes as the way of the original version, See pose_dataset.py and pose_augment.py. This includes scaling, rotation, flip, cropping.

This process can be a bottleneck for training, so if you have enough computing resources, please see Run for Faster Training Section

Run

$ python3 train.py --model=cmu --datapath={datapath} --batchsize=64 --lr=0.001 --modelpath={path-to-save}

2017-09-27 15:58:50,307 INFO Restore pretrained weights...

Run for Faster Training

If you have enough computing resources in multiple nodes, you can launch multiple workers on nodes to help data preparation.

worker-node1$ python3 pose_dataworker.py --master=tcp://host:port
worker-node2$ python3 pose_dataworker.py --master=tcp://host:port
worker-node3$ python3 pose_dataworker.py --master=tcp://host:port
...

After above preparation, you can launch training script with special arguments.

$ python3 train.py --remote-data=tcp://0.0.0.0:port

2017-09-27 15:58:50,307 INFO Restore pretrained weights...

Also, You can quickly train with multiple gpus. This automatically splits batch into multiple gpus for forward/backward computations.

$ python3 train.py --remote-data=tcp://0.0.0.0:port --gpus=8

2017-09-27 15:58:50,307 INFO Restore pretrained weights...

I trained models within a day with 8 gpus and multiple preprocessing nodes with 48 core cpus.

References

OpenPose

[1] https://github.com/CMU-Perceptual-Computing-Lab/openpose

[2] Training Codes : https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimation

[3] Custom Caffe by Openpose : https://github.com/CMU-Perceptual-Computing-Lab/caffe_train

Mobilenet

[1] Original Paper : https://arxiv.org/abs/1704.04861

[2] Pretrained model : https://github.com/tensorflow/models/blob/master/slim/nets/mobilenet_v1.md

Libraries

[1] Tensorpack : https://github.com/ppwwyyxx/tensorpack