One drawback of our previous closed-loop evaluation is that the simulator uses the same 2D perspective augmentation techniques present in the training dataset. We believe that there is a possibility that the bending objects may guide the policy to recover. We investigate this information leakage in our improved simulator by performing perspective augmentations that use the depth information and inpainting to avoid introducing artifacts in the evaluation procedure.
- Pipeline
- 3D perspective augmentations
- 2D perspective augmentations
mkdir raw_dataset- Download the UBP dataset into the "raw_dataset" directory. A sample of the UPB dataset is available here.
mkdir scene_splits- Download the scene splits into the "scene_splits" directory. The train-validation split is available here. In the "scene_splits" directory you should have: "train_scenes.txt" and "test_scenes.txt".
mkdir ckpts-
Train models using this repo
-
Copy the folders inside the snapshots dir into the ckpts dir.
mkdir -p pipeline/models/monodepth
cd pipeline/models/monodepthFor monodepth, download the pre-trained models from here
mkdir -p pipeline/models/inpaint
cd pipeline/models/inpaintFor the inpaint, download the pre-trained model from here
- Train models using this repo.
mkdir ckpts- The trained models are in the "snapshots" folder (in the above repo). Copy the directories from the "snapshots" folder to "ckpts" folder.
./multiple_runs.shpython3 view.py
