dxyang/ROT

Code for Watch and Match: Supercharging Imitation with Regularized Optimal Transport

Watch and Match: Supercharging Imitation with Regularized Optimal Transport

This is a repository containing the code for the paper "Watch and Match: Supercharging Imitation with Regularized Optimal Transport".

Download expert demonstrations, weights and environment libraries [link]

The link contains the following:

• The expert demonstrations for all tasks in the paper.
• The weight files for the expert (DrQ-v2) and behavior cloning (BC).
• The supporting libraries for environments (Gym-Robotics, metaworld) in the paper.
• Extract the files provided in the link
  • set the path/to/dir portion of the root_dir path variable in cfgs/config.yaml to the path of the ROT repository.
  • place the expert_demos and weights folders in ${root_dir}/ROT.

Instructions

• Install Mujoco based on the instructions given here.
• Install the following libraries:

sudo apt update
sudo apt install libosmesa6-dev libgl1-mesa-glx libglfw3

• Install dependencies

  • Set up Environment

  conda env create -f conda_env.yml
  conda activate rot
  

  • Install Gym-Robotics

  pip install -e /path/to/dir/Gym-Robotics
  

  • Install Meta-World

  pip install -e /path/to/dir/metaworld
  

  • Install particle environment (for experiment in Fig. 2 in the paper)

  pip install -e /path/to/dir/gym-envs
  

• Train BC agent - We provide three different commands for running the code on the DeepMind Control Suite, OpenAI Robotics Suite and the Meta-World Benchmark

  • For pixel-based input

  python train.py agent=bc suite=dmc obs_type=pixels suite/dmc_task=walker_run num_demos=10
  

  python train.py agent=bc suite=openaigym obs_type=pixels suite/openaigym_task=fetch_reach num_demos=50
  

  python train.py agent=bc suite=metaworld obs_type=pixels suite/metaworld_task=hammer num_demos=1
  

  python train_robot.py agent=bc suite=robot_gym obs_type=pixels suite/robotgym_task=reach num_demos=1
  

  • For state-based input

  python train.py agent=bc suite=dmc obs_type=features suite/dmc_task=walker_run num_demos=10
  

  python train.py agent=bc suite=openaigym obs_type=features suite/openaigym_task=fetch_reach num_demos=50
  

  python train.py agent=bc suite=metaworld obs_type=features suite/metaworld_task=hammer num_demos=1
  

• Train ROT - We provide three different commands for running the code on the DeepMind Control Suite, OpenAI Robotics Suite and the Meta-World Benchmark

  • For pixel-based input

  python train.py agent=potil suite=dmc obs_type=pixels suite/dmc_task=walker_run load_bc=true bc_regularize=true num_demos=10
  

  python train.py agent=potil suite=openaigym obs_type=pixels suite/openaigym_task=fetch_reach load_bc=true bc_regularize=true num_demos=50
  

  python train.py agent=potil suite=metaworld obs_type=pixels suite/metaworld_task=hammer load_bc=true bc_regularize=true num_demos=1
  

  python train_robot.py agent=potil suite=robotgym obs_type=pixels suite/robotgym_task=reach load_bc=true bc_regularize=true num_demos=1
  

  • For state-based input

  python train.py agent=potil suite=dmc obs_type=features suite/dmc_task=walker_run load_bc=true bc_regularize=true num_demos=10
  

  python train.py agent=potil suite=openaigym obs_type=features suite/openaigym_task=fetch_reach load_bc=true bc_regularize=true num_demos=50
  

  python train.py agent=potil suite=metaworld obs_type=features suite/metaworld_task=hammer load_bc=true bc_regularize=true num_demos=1
  

• Monitor results

tensorboard --logdir exp_local

Bibtex

@article{haldar2022watch,
         title={Watch and Match: Supercharging Imitation with Regularized Optimal Transport},
         author={Haldar, Siddhant and Mathur, Vaibhav and Yarats, Denis and Pinto, Lerrel},
         journal={CoRL},
         year={2022}
        }