This repository contains the sample code for the paper MobileHand: Real-time 3D Hand Shape and Pose Estimation from Color Image
Paper | Video | Results on STB Dataset B1 Random | Results on STB Dataset B1 Counting
If you find our code or paper useful, please consider citing
@inproceedings{MobileHand:2020,
title = {MobileHand: Real-time 3D Hand Shape and Pose Estimation from Color Image},
author = {Guan Ming, Lim and Prayook, Jatesiktat and Wei Tech, Ang},
booktitle = {27th International Conference on Neural Information Processing (ICONIP)},
year = {2020}
}
The simplest way to run our implementation is to use anaconda.
You can create an anaconda environment called mobilehand
with
conda env create -f environment.yaml
conda activate mobilehand
Next, you will need to download the MANO right hand model
- Go to MANO project page
- Click on Sign In and register for your account
- Download Models & Code (
mano_v1_2.zip
) - Unzip and copy the file
mano_v1_2/models/MANO_RIGHT.pkl
into themobilehand/model
folder
To allow the use of MANO model in Python 3 environment, we will need to remove Chumpy objects from the original MANO_RIGHT.pkl
model. The following steps are adapted from smplx repo:
conda create -n py27 python=2.7
conda activate py27
pip install chumpy
pip install tqdm
- Run the following command to remove any Chumpy objects and it will create a new file
MANO_RIGHT_NEW.pkl
:
python model/clean_ch.py --input-models model/MANO_RIGHT.pkl --output-folder model/
Change directory to the folder mobilehand/code/
cd code/
To test on a sample image from the STB dataset run:
python demo.py --dataset stb
To test on a sample image from the FreiHAND dataset run:
python demo.py --dataset freihand
To test on a sample video.mp4 file run:
python realtime.py
To test from your own camera or video file, you can uncomment/edit lines 24 and 25 of realtime.py
Jiawei Zhang, Jianbo Jiao, Mingliang Chen, Liangqiong Qu, Xiaobin Xu, and Qingxiong Yang
[ICCV 2019] FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images. [PDF] [Project] [Code]
Christian Zimmermann, Duygu Ceylan, Jimei Yang, Bryan Russell, Max Argus, Thomas Brox
[CVPR 2019] Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Rendering. [PDF]
Seungryul Baek, Kwang In Kim, Tae-Kyun Kim
Adnane Boukhayma, Rodrigo de Bem, Philip H.S. Torr
[CVPR 2019] 3D Hand Shape and Pose Estimation from a Single RGB Image. [PDF] [Project] [Code] (Oral)
Liuhao Ge, Zhou Ren, Yuncheng Li, Zehao Xue, Yingying Wang, Jianfei Cai, Junsong Yuan
[CVPR 2019] Learning joint reconstruction of hands and manipulated objects. [PDF] [Code] [Code] [Project]
Yana Hasson, Gül Varol, Dimitris Tzionas, Igor Kalevatykh, Michael J. Black, Ivan Laptev, and Cordelia Schmid
Xiong Zhang*, Qiang Li*, Wenbo Zhang, Wen Zheng
[CVPR 2020] Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild. [PDF] [Project] (Oral)
Dominik Kulon, Riza Alp Güler, Iasonas Kokkinos, Michael Bronstein, Stefanos Zafeiriou
[CVPR 2020] Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data. [PDF] [Project] [Code]
Yuxiao Zhou, Marc Habermann, Weipeng Xu, Ikhsanul Habibie, Christian Theobalt, Feng Xu
Filippos Gouidis, Paschalis Panteleris, Iason Oikonomidis, Antonis Argyros
Angjoo Kanazawa, Michael J Black, David W. Jacobs, Jitendra Malik
[SIGGRAPH ASIA 2017] Embodied Hands:Modeling and Capturing Hands and Bodies Together. [PDF] [Project]
Javier Romero, Dimitrios Tzionas, Michael J Black