Sai Kumar Dwivedi, Cordelia Schmid, Hongwei Yi, Michael J. Black, Dimitrios Tzionas
International Conference on 3D Vision (3DV 2024)
Clone the repository:
git clone https://github.com/saidwivedi/POCO.gitCreate fresh conda environment and install all the dependencies:
conda create -n poco python=3.8
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge
pip install -r requirements.txt
To run the demo, please download the pretrained models and necessary files from here. Note that the Downloads url works only after sign in -- you need to register first and agree with our license.
After downloading, unzip the file and ensure that the contents are placed in a folder named ./data.
We provide two versions of POCO: POCO-PARE and POCO-CLIFF. Note that POCO-CLIFF is more suitable for in-the-wild scenarios. To run a specific model, make sure to change the --cfg and --ckpt parameters accordingly i.e pare for POCO-PARE and cliff for POCO-CLIFF.
python demo.py --mode video --vid_file demo_data/friends.mp4 --cfg configs/demo_poco_cliff.yaml --ckpt data/poco_cliff.pt --output_folder out
python demo.py --mode folder --image_folder demo_data/images --cfg configs/demo_poco_cliff.yaml --ckpt data/poco_cliff.pt --output_folder out
Parts of the code are taken or adapted from the following repos:
We thank Partha Ghosh and Haiwen Feng for insightful discussions, Priyanka Patel for the CLIFF implementation, and Peter Kulits, Shashank Tripathi, Muhammed Kocabas, and the Perceiving Systems department for their feedback. SKD acknowledges support from the International Max Planck Research School for Intelligent Systems (IMPRS-IS). This work was partially supported by the German Federal Ministry of Education and Research (BMBF): Tubingen AI Center, FKZ: 01IS18039B.
If you find this code useful for your research, please consider citing the following paper:
@inproceedings{dwivedi_3dv2023_poco,
title={{POCO}: {3D} Pose and Shape Estimation using Confidence},
author={Dwivedi, Sai Kumar and Schmid, Cordelia and Yi, Hongwei and Black, Michael J. and Tzionas, Dimitrios},
booktitle={International Conference on 3D Vision (3DV)},
year={2024},
}This code is available for non-commercial scientific research purposes as defined in the LICENSE file. By downloading and using this code you agree to the terms in the LICENSE. Third-party datasets and software are subject to their respective licenses.
For code related questions, please contact sai.dwivedi@tuebingen.mpg.de
For commercial licensing (and all related questions for business applications), please contact ps-licensing@tue.mpg.de.