Garment4D

[PDF] | [OpenReview] | [arXiv] | [Project Page]

1. Overview

This is the codebase for our NeurIPS 2021 paper Garment4D: Garment Reconstruction from Point Cloud Sequences.

For further information, please contact Fangzhou Hong.

2. News

2021-12 Code release!
2021-09 Garment4D is accepted to NeurIPS 2021.

3. Getting Started

3.1 Setup the Environment

The code has been tested with Python 3.7.9, PyTorch 1.5.0, CUDA 10.1. Other required packages include:

tqdm
loguru
yaml
easydict
numpy
scipy
chamferdist
openmesh
pytorch_scatter

Other than the above packages, please also compile the cuda kernels required by PointNet++ by executing python setup.py install under modules/pointnet2/pointnet2.

3.2 Download SMPL model files

Please register and download SMPL model files here. Then please put the model files at smplx/model. The folder structure should be like

./
├── 
├── ...
└── smplx/
    ├── models/
        ├── smpl/
            ├── SMPL_MALE.pkl
            ├── SMPL_FEMALE.pkl
            └── SMPL_NEUTRAL.pkl
    ├── ...

3.3 Setup the Dataset

Please download the processed CLOTH3D dataset using the following links. Due the size of the whole dataset is big (~160 GB), we split the archieve into several 10 GB chunks. You could combine them by cat CLOTH3D.z* > merged_CLOTH3D.zip and unzip it and put it under dataset.


CLOTH3D.z01	CLOTH3D.z02	CLOTH3D.z03	CLOTH3D.z04
CLOTH3D.z05	CLOTH3D.z06	CLOTH3D.z07	CLOTH3D.z08
CLOTH3D.z09	CLOTH3D.z10	CLOTH3D.z11	CLOTH3D.z12
CLOTH3D.z13	CLOTH3D.z14	CLOTH3D.z15	CLOTH3D.zip

The folder structure should look like

./
├── 
├── ...
└── dataset/
    ├──CLOTH3D/
        ├── CLOTH3D/
        ├── CLOTH3D_template/
        ├── list/
        ├── Skirt_reg/
        ├── Trousers_reg/
        └── Tshirt_reg/

3.4 Download the Pre-train Models

Please download the pre-train models using the following links and put them under pretrain.

Pre-train Model	Download Link	L2 Error on CLOTH3D (Posed) [mm]
skirt.ckpt	link	49.23
trousers.ckpt	link	25.46
tshirt.ckpt	link	37.95

The folder structure should look like

./
├── 
├── ...
└── pretrain/
    ├── skirt.ckpt
    ├── trousers.ckpt
    └── tshirt.ckpt

3.5 Test with Pre-train models

Please checkout the scripts/test folder for the testing scripts. One should be able to run the corresponding testing scripts with the pre-train models and dataset setup correctly.

3.6 Train from Scratch

Please checkout the scripts/train folder for the training scripts. We currently support three types of garments i.e. skirts, Tshirts and trousers. Take skirts training as an example, please run the train_skirt_canonical.sh first for the canonical garment reconstruction and then run the train_skirt_posed.sh for the posed garment reconstruction.

4. License

Distributed under the MIT License. See LICENSE for more information.

5. Citation

If you find our work useful in your research, please consider citing the following papers:

@inproceedings{
    hong2021garmentd,
    title={Garment4D: Garment Reconstruction from Point Cloud Sequences},
    author={Fangzhou Hong and Liang Pan and Zhongang Cai and Ziwei Liu},
    booktitle={Thirty-Fifth Conference on Neural Information Processing Systems},
    year={2021},
    url={https://openreview.net/forum?id=aF60hOEwHP}
}

Acknowledgments

In our implementation, we refer to the following open-source databases:

NelsonBilber/Garment4D