/neugraspnet

NeuGraspNet: Learning Any-View 6DoF Robotic Grasping in Cluttered Scenes via Neural Surface Rendering

Primary LanguagePython

NeuGraspNet: Learning Any-View 6DoF Robotic Grasping in Cluttered Scenes via Neural Surface Rendering

Authors: Snehal Jauhri, Ishikaa Lunawat, and Georgia Chalvatzaki
Institution: PEARL Lab, TU Darmstadt, Germany
Published at: Robotics: Science and Systems, 2024

Project Site: https://sites.google.com/view/neugraspnet
Paper: https://arxiv.org/pdf/2306.07392

Release timeline:

  • 11th July 2024: Initial release with pre-trained weights and simulated grasping demos
  • September 2024: Dataset generation
  • October 2024: Training, Inference & ROS package

Installation

Tested on Ubuntu 20.04 with an NVIDIA GPU (Recommended 8GB GPU VRAM or higher)

With environment.yml:

  • Create a conda environment using the provided environment.yml file: 
    cd <this repo>
conda env create -f environment.yml

Or with manual conda installation:

  • Create a new conda environment with Python 3.8 or higher: 
    conda create --name neugraspnet python=3.8
  • Install requirements:
    (Due to compatibility issues with newer versions of open3d, sklearn installation needs to be enabled:) 
    conda activate neugraspnet
export SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True
pip install -r requirements.txt
  • Install torch-scatter based on pytorch version and cuda version (https://github.com/rusty1s/pytorch_scatter). For example: 
    pip install torch==1.13.0 torch-scatter==2.1.0 torchvision==0.14.0 -f https://data.pyg.org/whl/torch-1.13.0+cu117.html
  • Install the neugraspnet package: 
    cd <this repo>
pip install -e .
  • Build the conv_occupancy_network dependency: 
    python neugraspnet/scripts/convonet_setup.py build_ext --inplace

Run:

  • To run evaluations on the pile object dataset from VGN, run: 
    cd neugraspnet/neugraspnet
python -u scripts/test/sim_grasp_multiple.py --num-view 1 --object_set pile/test --scene pile --num-rounds 100 --model ./data/networks/neugraspnet_pile_efficient.pt --resolution=64 --type neu_grasp_pn_deeper_efficient --qual-th 0.5 --max_grasp_queries_at_once 40 --result-path ./data/results/neu_grasp_pile_efficient --sim-gui
    Modify the max_grasp_queries_at_once command line arguement based on your available GPU memory. (For eg. If using an RTX 3090, use max_grasp_queries_at_once= 40 or 60)
  • To run evaluations on the egad object dataset (https://dougsm.github.io/egad/), run: 
    python -u scripts/test/sim_grasp_multiple.py --num-view 1 --object_set egad --scene egad --num-rounds 100 --model ./data/networks/neugraspnet_pile_efficient.pt --resolution=64 --type neu_grasp_pn_deeper_efficient --qual-th 0.5 --max_grasp_queries_at_once 40 --result-path ./data/results/neu_grasp_egad_efficient --sim-gui

Acknowledgements: