🚀 Real Time Pose Estimation in TensorLayer
zsdonghao opened this issue · 9 comments
zsdonghao commented
A discussion for real-time (multi-person) pose estimation using TensorLayer and TensorFlow
Paper List
- 🚀Pose Proposal Networks, ECCV 2018
- DensePose
- 🚀 Convolutional Pose Machines
- Stacked Hourglass Networks for Human Pose Estimation
- 🚀Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
- VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera
- For Model Acceleration, see #416
Existing code/resource
- 🚀the author of
"Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields"
provides some links in his github: ZheC/Realtime_Multi-Person_Pose_Estimation, [video], [training log] - pyopenpose PyOpenPose
- 🚀COCO API for preparing the data - cocoapi ,
pip3 install pycocotools
, see keras_Realtime_Multi-Person_Pose_Estimation - tf-pose-estimation using MobileNet , video , blog🚀 , faster and other implementations
- data augmentation in cpp , see rmpe_dataset_server
- evaluate MPII result , see mpii-evluation
- openpose output format , see openpose
- Receptive Fields Calculator
Blog
- https://arvrjourney.com/human-pose-estimation-using-openpose-with-tensorflow-part-1-7dd4ca5c8027
- https://arvrjourney.com/human-pose-estimation-using-openpose-with-tensorflow-part-2-e78ab9104fc8
About Data Augmentation
- https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimation/blob/master/training/setLayers.py#L457
- https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimation/blob/master/training/setLayers.py#L343
Others
- MPII benchmarks http://human-pose.mpi-inf.mpg.de/#related_benchmarks
- TensorFlow lite
- 知乎 https://www.zhihu.com/question/59750782/answer/207238452
- CSDN CPM http://blog.csdn.net/shenxiaolu1984/article/details/51094959
- Use TF dataset API to deal with large dataset https://blog.csdn.net/dQCFKyQDXYm3F8rB0/article/details/79342369
zsdonghao commented
I release the APIs to download or visualise MPII dataset in 1 line of code here: #482
import pprint
import tensorlayer as tl
img_train_list, ann_train_list, img_test_list, ann_test_list = tl.files.load_mpii_pose_dataset()
print(img_train_list[0])
pprint.pprint(ann_train_list[0])
for i in range(100): # show 100 images
image = tl.vis.read_image(img_train_list[i])
tl.vis.draw_mpii_people_to_image(image, ann_train_list[i], '_temp%d.png' % i)
zsdonghao commented
Any ideas to make it run faster?
Algorithm
OpenPose | Idea | |
---|---|---|
image size | 368x654 i.e. 9:16 | 244x244 ? |
CNN | VGG-19 | mobilenetV2 or others, see #416 |
CNN | Residual Squeeze | (1), (2) |
Engineering
- Faster pre-processing : ❓
- Faster post-processing :
- @filipetrocadoferreira c++/cython multiprocessing module
- pypy vs cython
- TensorRT
filipetrocadoferreira commented
I think where you can achieve better results easier is in post-processing. At least in tf-pose-estimation that's the current bottleneck. maybe implementing a c++/cython multiprocessing module for that can be enough
1icas commented
I'm trying to implement a real time pose estimation(mulit-person) on mobile. For the mobile performence, I'm trying to quantize the model.
1icas commented
@zsdonghao yes,i know. But it's not satisfy the performance requirement. Also i'm very interested in quantisation.
1icas commented
Or do you have other suggesstion for model acceleration ?