Repo for "TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinement, ECCV'22 (Poster)"
[Paper] [Project Page]
We recommend running the code in conda environment:
conda create -n toch python=3.7
conda activate toch
Clone the repository and install main dependencies with
git clone https://github.com/kzhou23/toch.git && cd toch
pip install -r requirements.txt
We additionally require the following libraries:
Please check the respective instructions for downloading and installation.
- Download the raw GRAB dataset and SMPL-X models by following instructions here.
- Clone the GRAB repository and copy its subfolder:
git clone https://github.com/otaheri/GRAB.git
shopt -s extglob
cp GRAB/tools/!(objectmodel.py) toch/data/grab/tools/
- Run our pre-processing code:
python data/grab/preprocessing.py --grab_path $RAW_GRAB_FOLDER \
--smplx_path $SMPLX_MODEL_FOLDER \
--mano_path $MANO_MODEL_FOLDER \
--out_path $PROCESSED_GRAB_FOLDER
python data/grab/compute_hand_obj_corr.py --grab_path $RAW_GRAB_FOLDER \
--data_path $PROCESSED_GRAB_FOLDER \
--mano_path $MANO_MODEL_FOLDER \
--num_proc 50
TODO
Train the model with
python train.py --num_gpu 3 --data_path $PROCESSED_GRAB_FOLDER
The model checkpoint will be saved under ./ckpt
by default. Feel free to explore the available training options.
You can refine a sequence from the pre-processed GRAB dataset with
python scripts/reconstruct_grab_seq.py --grab_path $RAW_GRAB_FOLDER \
--ckpt_path $PRETRAINED_MODEL_PATH \
--mano_path $MANO_MODEL_FOLDER \
--seq_path $INPUT_SEQUENCE_PATH
The output meshes will be saved under ./recon_results
by default.
TODO
@inproceedings{zhou2022toch,
title = {TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement},
author = {Zhou, Keyang and Bhatnagar, Bharat Lal and Lenssen, Jan Eric and Pons-Moll, Gerard},
booktitle = {European Conference on Computer Vision ({ECCV})},
month = {October},
organization = {{Springer}},
year = {2022},
}