ZeDO: Back to Optimization: Diffusion-based Zero-Shot 3D Human Pose Estimation

This is the official implementation of this paper:

Zhongyu Jiang, Zhuoran Zhou, Lei Li, Wenhao Chai, Cheng-Yen Yang, and Jenq-Neng Hwang. Back to Optimization: Diffusion-based Zero-Shot 3D Human Pose Estimation arXiv preprint arXiv:2307.03833 (2023).

Results on 3DPW

Under PA-MPJPE and MPJPE

Methods	CE	Opt	PA-MPJPE ↓	MPJPE ↓
SPIN			59.2	96.9
VIBE			51.9	82.9
PARE			46.4	74.7
HybrIK			45.0	74.1
HybrIK	✓		50.9	82.0
PoseAug	✓		58.5	94.1
AdaptPose	✓		46.5	81.2
PoseDA	✓		55.3	87.7
ZeDO (J=17)	✓	✓	42.6	80.9
ZeDO (J=14)	✓	✓	45.4	88.6

Evaluate model

Envrionment Setup

pytorch >= 1.10

conda create -n ZeDO python==3.9
conda activate ZeDO
pip install -r requirements.txt

Data and Model Preparation

Evaluation dataset, clusters and checkpoint: Google Drive

${POSE_ROOT}
|-- configs
|-- lib
|-- run
|-- checkpoint
    |-- concatebb
        |-- checkpoint_1500.pth
|-- data
    |-- h36m
        |-- h36m_test.pkl
        |-- h36m_sh_dt_ft.pkl
    |-- 3dpw
        |-- pw3d_test.npz
    |-- 3dhp
        |-- mpii3d_test.pkl
|-- clusters
    |-- 3dhp_cluster1.pkl
    |-- h36m_cluster1.pkl
    |-- 3dhp_cluster50.pkl
    |-- h36m_cluster50.pkl

Evaluation script

python -m run.opt_main --config configs/subvp/concat_pose_optimization_<dataset>.py --ckpt_dir ./checkpoint/concatebb --ckpt_name checkpoint_1500.pth  --hypo 1 <--gt>

Citation

If you find this code useful in your project, please consider citing:

@article{jiang2023back,
  title={Back to Optimization: Diffusion-based Zero-Shot 3D Human Pose Estimation},
  author={Jiang, Zhongyu and Zhou, Zhuoran and Li, Lei and Chai, Wenhao and Yang, Cheng-Yen and Hwang, Jenq-Neng},
  journal={arXiv preprint arXiv:2307.03833},
  year={2023}
}

Acknowledgement

This repo is built on the excellent work score_sde by Yang Song and GFPose by Hai Ci.

haorui-ji/ZeDO-Release