In recent years, tremendous amount of progress is being made in the field of Human Pose Estimation from RGB Camera, which is an interdisciplinary field that fuses computer vision, deep/machine learning and anatomy. This repo is for my study notes and will be used as a place for triaging new research papers.
To make it a collaborative project, you may add content through pull requests or open an issue to let me know.
The projects and papers are sorted by season, then by month, then by alphabet. I do this because newer papers are usually more sophisticated than older ones and to make removing duplicates easier. Below the publications are short, memorable bite sized summaries. These summaries should be understandable to someone who is four, and very short.
- Time Dimension
- 📷 Single-Shot
- 🎥 Video/Real-Time
- Spatial Dimensions
- 🚪 2D Models
- 📦 3D Models
📷📦Deep 3D Human Pose Estimation Under Partial Body Presence (Oct 2018)
My legs have been chopped off, and my head, but tis but a scratch.
📷📦3D Human Pose Estimation Using Stochastic Optimization In Real Time (Oct 2018)
Try again and again, till you get it right. Uses depth based sensors. RTW + Particle Swarms
📷📦Adversarial 3D Human Pose Estimation via Multimodal Depth Supervision (Sep 2018)
Continuation of FBI work, also got multimodal network now. IDK What that means
📷🚪Deeply Learned Compositional Models for Human Pose Estimation (Sep 2018)
Take the human body, and shove it into code blocks
🚪Dense Pose Transfer (Sep 2018)
color in a mannequin, a machine imagines the details, then animates it's paper statue
📷📦3D Ego-Pose Estimation via Imitation Learning (Sep 2018)
Headcam, they use a very complicated ragdoll, also, just walking and running
📷📦3D Human Pose Estimation with Siamese Equivariant Embedding (Sep 2018)
Twins compare their answers after doing their math homework. (Homework refers to the 3d pose estimation)
They block their face and body with cheap photoshop techniques, then the machine has to "x-ray" through all that.
📷📦Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimation [CODE] (Aug 2018)
The circle of 3D pose estimation. 2d Image -> 2d Color Me Rad guy -> 3d pudgy man -> photograph of said man -> 2d Image
📷📦 Single-Shot Multi-Person 3D Body Pose Estimation From Monocular RGB Input (Aug 2018)
They use a ORPM, whatever that means. And they have some very obviouly green screened images.
📷📦 Rethinking Pose in 3D: Multi-stage Refinement and Recovery for Markerless Motion Capture (Aug 2018)
use lots of cameras to make just one camera better, and do this over and over and over again
📷📦3D Human Pose Estimation with Relational Networks (Jul 2018)
Back bone connected to the shoulder bone, shoulder bone connected to the neck bone...
🚪Human Pose Estimation with Parsing Induced Learner (Jun 2018)
📷📦FBI-Pose: Towards Bridging the Gap between 2D Images and 3D Human Poses using Forward-or-Backward Information (Jun 2018)
Anderson Silva's broken bent leg
📷📦DRPose3D: Depth Ranking in 3D Human Pose Estimation (May 2018)
These guys do FBI but without the "crowd sourced" annotations
📷📦It's all Relative: Monocular 3D Human Pose Estimation from Weakly Supervised Data (May 2018)
Crowd sourced relative depth annotations
📦BodyNet: Volumetric Inference of 3D Human Body Shapes [CODE] (Apr 2018)
Make a statue of michealangelo in minecraft
📦Unsupervised Geometry-Aware Representation for 3D Human Pose Estimation [CODE] (Apr 2018)
It rotates the person with it's eyes. (Insert reference to undressing with eyes here)
🎥📦 MonoPerfCap: Human Performance Capture from Monocular Video [Project] (Mar 2018)
makes a 3d replica of you like a fully featured action figure
📦Learning to Estimate 3D Human Pose and Shape from a Single Color Image (May 2018)
SMPL brand Artist's Mannequin
📷📦 3D Human Pose Estimation in the Wild by Adversarial Learning (Mar 2018)
🎥📦 LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images [Project] (Mar 2018)
📷📦 Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations [Project page] (Mar 2018)
🎥📦 End-to-end Recovery of Human Shape and Pose [CODE] (Dec 2017)
📷📦Exploiting temporal information for 3D human pose estimation (Nov 2017)
📷📦 DensePose: Dense Human Pose Estimation In The Wild [CODE] [Project page] (Feb 2018)
🎥🚪 Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields [CODE] (Apr 2017)
📷🚪 Adversarial PoseNet: A Structure-aware Convolutional Network for Human Pose Estimation (May 2017)
📷📦 A simple yet effective baseline for 3d human pose estimation (Aug 2017) [CODE]
🎥📦 VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera [CODE] [Project] (Jul 2017)
📷📦 Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image (Oct 2017)
📷📦Knowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation [CODE] (Aug 2017)
📷📦 Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation (Nov 2016)
📷📦 Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision [Project] (Nov 2016)
📷📦 MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild (Oct 2016)
📷📦3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information (Sep 2016)
📷📦 Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image (Jul 2016)
📷🚪 Stacked Hourglass Networks for Human Pose Estimation [CODE] (Mar 2016)
📷🚪 Convolutional Pose Machines [CODE] (Jan 2016)
🎥📦 Spatio-temporal Matching for Human Pose Estimation [Project] (Dec 2015)
🎥📦 Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video [Project] (Nov 2015)
MS COCO
MPII POSE
Human 3.6M
Human Eva
MPI INF 3DHP
Unite The People
Pose Guided Person Image Generation
A Generative Model of People in Clothing
Deformable GANs for Pose Based Human Image Generatoin
Dense Pose Transfer
Pose Guided Person Image Generation - [CODE] - Ma, L., Jia, X., Sun, Q., Schiele, B., Tuytelaars, T., & Gool, L.V. (NIPS 2017)
A Generative Model of People in Clothing - Lassner, C., Pons-Moll, G., & Gehler, P.V. (ICCV 2017)
Deformable GANs for Pose-based Human Image Generation - [CODE] - Siarohin, A., Sangineto, E., Lathuilière, S., & Sebe, N. (CVPR 2018)
Dense Pose Transfer - Neverova, N., Guler, R.A., & Kokkinos, I. (ECCV 2018)
Gesture and Sign Language Recognition with Deep Learning
-
I'd like to find a project I can clone.
-
I'd like to find a recent project.
-
I'd like to find a project with the 3d work done
-
I'd like to find a project that can integrate with SteamVR. (bone locations instead of blobs/meshes)
-
state of the art 2d pose detector, this is crucial.
-
This 2d pose detector can return colored limbs corresponding to each body part or heat maps corresponding to joint, or even forward/backwards information from annoation.
-
Time dependence ideally the network, when doing pixel to 2d pose and 2d pose to 3d map and 3d map to skeleton should take into account the previous frame, and with an internal representation of the boundary conditions of the human body pose (Perhaps a GAN?)
-
Additionally, there are other gimmicks that can be used, like Siamese network parallelism. Where two shots of the same pose are rewarded for giving the same output. Physics simulations of body mechanics can be used. Reprojection of 3d joints back to 2d geometery using meshes. The 3d pose can be iteratively refined over and over again. Sythetic data created by game data. Additionally there should be 3d pose standarization. Additionally 2d pose should be done well.
Domain Transfer for 3D Pose Estimation from Color Images without Manual Annotations (Oct 2018)
Not relevant, but hand posing is here, so I guess it kinda is
Context-Aware Deep Spatio-Temporal Network for Hand Pose Estimation from Depth Images (Oct 2018)
Not relvant, hand pose estimation
📷📦Cascaded Pyramid Network for 3D Human Pose Estimation Challenge (Oct 2018)
Top down, seems pretty typical, nothing special goin on here
📷📦 Deep Textured 3D Reconstruction of Human Bodies [Project] (Sep 2018)
Not relevant. I'll make a replica out of you from soggy clay. Depth training, regular camera tests
**:camera:
Multiview 3D human pose estimation using improved least-squares and LSTM networks (Jul 2018) LSTM
Hierarchical Contextual Refinement Networks for Human Pose Estimation (Oct 2018)
You start from your center and work outwards
Fully Automatic Multi-person Human Motion Capture for VR Applications (Sep 2018)
$$$ Have a party, turn all your friends into skeletons then they leave
Propagating LSTM: 3D Pose Estimation based on Joint Interdependency (Sep 2018)
[Hockey Pose Estimation and Action Recognition using Convolutional Neural Networks to Ice Hockey][https://uwspace.uwaterloo.ca/handle/10012/13835] (Sep 2018)
Human pose estimation method based on single depth image (Sep 2018)
A Review of Human Pose Estimation from Single Image (Jul 2018)
3D Human pose estimation on Taiji sequence (Jul 2018)
MoCap and a new biomedical dataset!
Human Pose Estimation Based on Deep Neural Network (Jul 2018)
Multi-View CNNs for 3D Hand Pose Estimation (Jul 2018)
Lots a people lookin at your hand
Multiview 3D human pose estimation using improved least-squares and LSTM networks (Jul 2018) LSTM
3-D Reconstruction of Human Body Shape from a Single Commodity Depth Camera (Jun 2018)
Stacked dense-hourglass networks for human pose estimation (Apr 2018)
🚪Simple Baselines for Human Pose Estimation and Tracking [CODE] (Apr 2018)
A generalizable approach for multi-view 3D human pose regression (Apr 2018)
A Deep Learning Based Method For 3D Human Pose Estimation From 2D Fisheye Images (Mar 2018)
A Unified Framework for Multi-View Multi-Class Object Pose Estimation (Mar 2018)
Learning Monocular 3D Human Pose Estimation from Multi-view Images (Mar 2018)
Multi-Scale Structure-Aware Network for Human Pose Estimation (Mar 2018)
Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera (Mar 2018)
Image-based Synthesis for Deep 3D Human Pose Estimation (Feb 2018)
🚪LSTM Pose Machines [CODE] (Dec 2017)
Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB (Dec 2017)
Using a single RGB frame for real time 3D hand pose estimation in the wild (Dec 2017)
📦Learning 3D Human Pose from Structure and Motion (Nov 2017)
📦Integral Human Pose Regression [CODE] (Nov 2017)
Human Pose Retrieval for Image and Video collections (Oct 2017)
A search engine for dancers
🚪Human Pose Estimation Using Global and Local Normalization (Sep 2017)
🚪Learning Feature Pyramids for Human Pose Estimation [CODE] (Aug 2017)
📦Recurrent 3D Pose Sequence Machines (Jul 2017)
🚪Self Adversarial Training for Human Pose Estimation [CODE1][CODE2] (Jul 2017)
Learning Human Pose Models from Synthesized Data for Robust RGB-D Action Recognition (Jul 2017)
Faster Than Real-time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses (Jul 2017)
[A Dual-Source Approach for 3D Human Pose Estimation from a Single Image] (https://arxiv.org/pdf/1705.02883.pdf) (May 2017)
📦Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach [CODE] (Apr 2017)
Adversarial PoseNet: A Structure-Aware Convolutional Network for Human Pose Estimation (Apr 2017)
Forecasting Human Dynamics from Static Images (Apr 2017)
📦Compositional Human Pose Regression (Apr 2017)
2D-3D Pose Consistency-based Conditional Random Fields for 3D Human Pose Estimation (Apr 2017)
🚪Multi-context Attention for Human Pose Estimation - [CODE] (Feb 2017)
📦Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image (Jan 2017)
🚪Towards Accurate Multi-person Pose Estimation in the Wild [CODE] (Jan 2017)
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image (Jan 2017)
Learning from Synthetic Humans (Jan 2017)
MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior (Jan 2017)
🚪RMPE: Regional Multi-person Pose Estimation [CODE1][CODE2] (Dec 2016)
📦Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose [CODE] (Nov 2016)
🚪Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields [CODE] (Nov 2016)
3D Human Pose Estimation from a Single Image via Distance Matrix Regression (Nov 2016)
Learning camera viewpoint using CNN to improve 3D body pose estimation (Sep 2016)
EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras (Sep 2016)
📦Structured Prediction of 3D Human Pose with Deep Neural Networks (May 2016)
🚪DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model [CODE1][CODE2] (May 2016)
🚪Recurrent Human Pose Estimation [CODE] (May 2016)
Synthesizing Training Images for Boosting Human 3D Pose Estimation (Apr 2016)
Seeing Invisible Poses: Estimating 3D Body Pose from Egocentric Video - Completely insane and above the scope of science (Mar 2016)
🚪DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation [CODE] (Nov 2015)
A Dual-Source Approach for 3D Pose Estimation from a Single Image (Sep 2015)
🚪Human Pose Estimation with Iterative Error Feedback [CODE] (Jul 2015)
🚪Flowing ConvNets for Human Pose Estimation in Videos [CODE] (Jun 2015)
📦3D Human Pose Estimation from Monocular Images with Deep Convolutional Neural Network (Nov 2014)
🚪Efficient Object Localization Using Convolutional Networks (Nov 2014)
🚪MoDeep: A Deep Learning Framework Using Motion Features for Human Pose Estimation (Sep 2014)
🚪Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation [CODE] (Jun 2014)
🚪Learning Human Pose Estimation Features with Convolutional Networks (Dec 2013)
🚪DeepPose: Human Pose Estimation via Deep Neural Networks (Dec 2013)
Deep 3D Pose Dictionary: 3D Human Pose Estimation from Single RGB Image Using Deep Convolutional Neural Network 3D Hand Pose Tracking from Depth Images using Deep Reinforcement Learning
Human 3D Reconstruction and Identification Using Kinect Sensor (Aug 2018) Low-fi body ID
3D Head Pose Estimation Using Tensor Decomposition and Non-linear Manifold Modeling (Sep 2018) `
` A Data-Driven Approach for 3D Human Body Pose Reconstruction from a Kinect Sensor Accidental Fall Detection Based on Pose Analysis and SVDD