/Garment4D

[NeurIPS 2021] Garment4D: Garment Reconstruction from Point Cloud Sequences

Primary LanguagePythonMIT LicenseMIT

Garment4D

[PDF] | [OpenReview] | [arXiv] | [Project Page]

1. Overview

This is the codebase for our NeurIPS 2021 paper Garment4D: Garment Reconstruction from Point Cloud Sequences.

teaser

For further information, please contact Fangzhou Hong.

2. News

  • 2021-12 Code release!
  • 2021-09 Garment4D is accepted to NeurIPS 2021.

3. Getting Started

3.1 Setup the Environment

The code has been tested with Python 3.7.9, PyTorch 1.5.0, CUDA 10.1. Other required packages include:

Other than the above packages, please also compile the cuda kernels required by PointNet++ by executing python setup.py install under modules/pointnet2/pointnet2.

3.2 Download SMPL model files

Please register and download SMPL model files here. Then please put the model files at smplx/model. The folder structure should be like

./
├── 
├── ...
└── smplx/
    ├── models/
        ├── smpl/
            ├── SMPL_MALE.pkl
            ├── SMPL_FEMALE.pkl
            └── SMPL_NEUTRAL.pkl
    ├── ...

3.3 Setup the Dataset

Please download the processed CLOTH3D dataset using the following links. Due the size of the whole dataset is big (~160 GB), we split the archieve into several 10 GB chunks. You could combine them by cat CLOTH3D.z* > merged_CLOTH3D.zip and unzip it and put it under dataset.

CLOTH3D.z01 CLOTH3D.z02 CLOTH3D.z03 CLOTH3D.z04
CLOTH3D.z05 CLOTH3D.z06 CLOTH3D.z07 CLOTH3D.z08
CLOTH3D.z09 CLOTH3D.z10 CLOTH3D.z11 CLOTH3D.z12
CLOTH3D.z13 CLOTH3D.z14 CLOTH3D.z15 CLOTH3D.zip

The folder structure should look like

./
├── 
├── ...
└── dataset/
    ├──CLOTH3D/
        ├── CLOTH3D/
        ├── CLOTH3D_template/
        ├── list/
        ├── Skirt_reg/
        ├── Trousers_reg/
        └── Tshirt_reg/

3.4 Download the Pre-train Models

Please download the pre-train models using the following links and put them under pretrain.

Pre-train Model Download Link L2 Error on CLOTH3D (Posed) [mm]
skirt.ckpt link 49.23
trousers.ckpt link 25.46
tshirt.ckpt link 37.95

The folder structure should look like

./
├── 
├── ...
└── pretrain/
    ├── skirt.ckpt
    ├── trousers.ckpt
    └── tshirt.ckpt

3.5 Test with Pre-train models

Please checkout the scripts/test folder for the testing scripts. One should be able to run the corresponding testing scripts with the pre-train models and dataset setup correctly.

3.6 Train from Scratch

Please checkout the scripts/train folder for the training scripts. We currently support three types of garments i.e. skirts, Tshirts and trousers. Take skirts training as an example, please run the train_skirt_canonical.sh first for the canonical garment reconstruction and then run the train_skirt_posed.sh for the posed garment reconstruction.

4. License

Distributed under the MIT License. See LICENSE for more information.

5. Citation

If you find our work useful in your research, please consider citing the following papers:

@inproceedings{
    hong2021garmentd,
    title={Garment4D: Garment Reconstruction from Point Cloud Sequences},
    author={Fangzhou Hong and Liang Pan and Zhongang Cai and Ziwei Liu},
    booktitle={Thirty-Fifth Conference on Neural Information Processing Systems},
    year={2021},
    url={https://openreview.net/forum?id=aF60hOEwHP}
}

Acknowledgments

In our implementation, we refer to the following open-source databases: