/deep-visual-constraints

Pytorch implimentation of the paper: "Deep Visual Constraints: Neural Implicit Models for Manipulation Planning from Visual Input"

Primary LanguageJupyter Notebook

Deep Visual Constraints (DVC)

This is a pytorch implimentation of the paper: "Deep Visual Constraints: Neural Implicit Models for Manipulation Planning from Visual Input".

[Project Page] [Paper] [Video]

Requirements

  • Pytorch
  • Torchvision
  • H5py
  • Trimesh
  • Scipy
  • Pyglet
  • Matplotlib
  • Scikit-image
  • Tqdm
  • Tensorboard

Instruction

  1. Download the pretrained network files into the folder './network'
  2. Download the dataset and extract it to the folder './data'
  3. Run 'visualize_*.ipynb' to visualize data, learend SDFs (& mesh reconstruction), PCAs on learned features, or tasks (optimized grasp/hang poses)
  4. Run 'train_PIFO.ipynb' to train the whole framework

Citation

@article{ha2022dvc,
  title={Deep Visual Constraints: Neural Implicit Models for Manipulation Planning from Visual Input},
  author={Ha, Jung-Su and Driess, Danny and Toussaint, Marc},
  journal={arXiv preprint arXiv:2112.04812},
  year={2022}
}