/SSCF

[SIGGRAPH 2024] Spatial and Surface Correspondence Field for Interaction Transfer

Primary LanguagePythonApache License 2.0Apache-2.0

Spatial and Surface Correspondence Field for Interaction Transfer

SIGGRAPH 2024

Zeyu Huang, Honghao Xu, Haibin Huang, Chongyang Ma, Hui Huang, Ruizhen Hu

This repository provides code, data and pretrained models for SSCF.

[Project Webpage] | [Paper]

Environment Setup

Installation

  1. Create a new conda env
conda create -n sscf python=3.9 cmake make compilers pybind11 eigen -c conda-forge
  1. Install pyigl
pip install git+https://github.com/zzilch/pyigl
# pip install git+https://gitee.com/zzilch/pyigl
  1. Install Pytorch and Pytorch3D based on your cuda version. (Our Pytorch version is 2.4.0 and cuda version is 12.2)
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=12.1 -c pytorch -c nvidia
conda install pytorch3d -c pytorch3d
  1. Install the human models with smplx and mano.
pip install smplx
pip install git+https://github.com/lixiny/manotorch.git
  1. Install other dependencies.
pip install trimesh[all] pyvista[all] point-cloud-utils h5py scikit-learn cycpd chumpy scikit-image matplotlib imageio plotly opencv-python open3d warp-lang

pip install numpy==1.23.0

Data Preparation

We directly provide the processed data and checkpoints, please download them and organize them as following:

sscf/
└── data/
    ├── ckpts/
    ├── hoi/
    ├── human_model/
    ├── shapenet/

For details on data preprocessing, please refer to preprocess_data.

Evaluation

Human-chair Interaction

If you want to evaluate our model on human-chair interaction, please run:

python generate_chairs_interaction.py

Hand-mug Interaction

If you want to evaluate our model on hand-mug interaction, please run:

python generate_mugs_interaction.py

Reproduction of baselines

We also provide our reproduction code of other baselines, please refer to baselines for more details.

Citation

If you find SSCF useful in your research, please cite our paper:

@article{InterTransfer24,
    title={Spatial and Surface Correspondence Field for Interaction Transfer},
    author={Zeyu Huang and Honghao Xu and Haibin Huang and Chongyang Ma and Hui Huang and Ruizhen Hu},
    journal={ACM Transactions on Graphics (Proceedings of SIGGRAPH)},
    volume={43},
    number={4},
    pages={83:1--83:12},
    year={2024},
}

Acknowledgment

We thank for the following excellent open source projects: