air23zj/universal_manipulation_interface

Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots

Universal Manipulation Interface

[Project page] [Paper] [Hardware Guide] [Data Collection Instruction] [SLAM repo] [SLAM docker]

Cheng Chi^1,2, Zhenjia Xu^1,2, Chuer Pan¹, Eric Cousineau³, Benjamin Burchfiel³, Siyuan Feng³,

Russ Tedrake³, Shuran Song^1,2

¹Stanford University, ²Columbia University, ³Toyota Research Institute

🛠️ Installation

Only tested on Ubuntu 22.04

Install docker following the official documentation and finish linux-postinstall.

Install system-level dependencies:

$ sudo apt install -y libosmesa6-dev libgl1-mesa-glx libglfw3 patchelf

We recommend Miniforge instead of the standard anaconda distribution for faster installation:

$ mamba env create -f conda_environment.yaml

Running UMI SLAM pipeline

Copy all videos for a data collection session into folder <session>.

Run SLAM pipeline

$ python run_slam_pipeline.py <session>

Generate dataset for training.

$ python scripts_slam_pipeline/07_generate_replay_buffer.py -o dataset.zarr.zip <session>

Training Diffusion Policy

$ python train.py --config-name=train_diffusion_unet_timm_umi_workspace task.dataset_path=dataset.zarr.zip

🚧 More Detailed Documentation Coming Soon! 🚧