Replication of "KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients"
This is a modified version of the original project to support the following:
- Compatible with CARLA 0.9.14
- Easy visualisation (gif generation)
- Minor fixes
- GPU: NVIDIA Tesla T4
- Memory: 16GB+
- Storage: 150GB+
- Ubuntu 20.04
- nvidia driver
Clone the repo
git clone https://github.com/ADS-Testing/king.git
cd king
Install drivers and reboot. If the appropriate version of the driver is already installed(Check with the command nvidia-smi
), you can skip this step.
sudo apt update
sudo apt install ubuntu-drivers-common
sudo ubuntu-drivers autoinstall
sudo reboot
Install anaconda and build the environment.
wget https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh
bash Anaconda3-2022.05-Linux-x86_64.sh
source ~/.profile
conda env create -f environment.yml
conda activate king
Download and setup CARLA 0.9.14.
chmod +x setup_carla.sh
./setup_carla.sh
To generate scenarios for TransFuser, you need to download the model weights:
mkdir -p driving_agents/king/transfuser/model_checkpoints/regular
cd driving_agents/king/transfuser/model_checkpoints/regular
wget https://s3.eu-central-1.amazonaws.com/avg-projects/transfuser/models.zip
unzip models.zip
rm -rf models.zip late_fusion geometric_fusion cilrs aim
cd -
We provide bash scripts for the experiments for convenience. Please make sure the "CARLA_ROOT" ("./carla_server" by default) and "KING_ROOT" (if present) environment variables are set correctly in all of those scripts.
For all of the generation scripts, first spin up a carla server in a separate shell:
carla_server/CarlaUE4.sh --world-port=2000 -RenderOffScreen
If you cannot separate the shell, execute the script in the background.
nohup carla_server/CarlaUE4.sh --world-port=2000 -RenderOffScreen &
Following scripts will run generation and automatically evaluate the results.
For AIM-BEV generation, run:
bash run_generation.sh
For AIM-BEV generation using both gradient paths, run:
bash run_generation_both_paths.sh
For Transfuser generation using both gradient paths, run:
bash run_generation_transfuser.sh
generation_results/
├── agents_1
│ ├── RouteScenario_112_to_112
│ │ ├── results.json
│ │ └── scenario_records.json
│ ├── RouteScenario_114_to_114
│ │ ├── results.json
│ │ └── scenario_records.json
│ ...
│ ├── opt.pkl
│ └── opt.txt
├── agents_2
│ ├── RouteScenario_112_to_112
│ │ ├── results.json
│ │ └── scenario_records.json
│ ...
│ ├── opt.pkl
│ └── opt.txt
└── agents_4
├── RouteScenario_112_to_112
│ ├── results.json
│ └── scenario_records.json
...
├── opt.pkl
└── opt.txt
First spin up a carla server in a separate shell:
carla_server/CarlaUE4.sh --world-port=2000 -RenderOffScreen
After providing the directory name you want to visualize as an argument, run the following script. The default directory is set to "generation_results".
bash run_visualization.sh generation_results_transfuser
generation_results_transfuser/
└── agents_4
├── RouteScenario_112_to_112
│ ├── RouteScenario_112_iter_4.gif
│ ├── results.json
│ └── scenario_records.json
├── RouteScenario_114_to_114
│ ├── RouteScenario_114_iter_1.gif
│ ├── results.json
│ └── scenario_records.json
...
├── opt.pkl
└── opt.txt
To fine-tune the original agent on KING scenarios, first download the regular data for AIM-BEV:
chmod +x download_regular_data.sh
./download_regular_data.sh
Then run the fine-tuning script (and adjust it for the correct dataset path, if necessary):
bash run_fine_tuning.sh
This script also automatically runs evaluation on KING scenarios.
To evaluate the original checkpoint for AIM-BEV on the Town10 intersections benchmark, spin up a carla server and run
leaderboard/scripts/run_evaluation.sh
To evaluate a fine-tuned model, run the fine-tuning script above and toggle the commented "TEAM_CONFIG" variable in the evaluation script to change the model weights.
- Carla terminates immediately
- CondaValueError: prefix already exists:
- ego_bp = self.bp_library.filter("vehicle.lincoln.mkz2017")[0] IndexError: index out of range
- error while loading shared libraries: libomp.so.5
- error: command 'gcc' failed with exit status 1
- Exception thrown: bind: Address already in use
- FileNotFoundError: [Errno 2] No such file or directory: 'aplay'
- ImportError: cannot import name 'TrafficLightState' from 'carla' (unknown location)
- ModuleNotFoundError: No module named 'agents.navigation.global_route_planner_dao'
- No space left on device
- Unable to install gcc
Related pages:
This implementation is based on code from several repositories.
@inproceedings{Hanselmann2022ECCV,
author = {Hanselmann, Niklas and Renz, Katrin and Chitta, Kashyap and Bhattacharyya, Apratim and Geiger, Andreas},
title = {KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients},
booktitle = {European Conference on Computer Vision(ECCV)},
year = {2022}
}
- Taehyeon An
- Kyungwook Nam
- Advised by Donghwan Shin