This repository contains the code for "Task-Aware Risk Estimation of Perception Failures for Autonomous Vehicles", Proceedings of Robotics: Science and Systems (RSS), 2023.
- Python 3.9
- Poetry >1.2 (you can install it following this guide)
To run the code you need to install the nuPlan dataset. At the moment only the mini version is required. Follow this guide to download and install the dataset.
This project relies on the environment variables to find the dataset.
In your ~/.bashrc
(or equivalent), add
export NUPLAN_DATA_ROOT="$HOME/datasets/nuplan/"
export NUPLAN_MAPS_ROOT="$HOME/datasets/nuplan/maps"
export NUPLAN_EXP_ROOT="$HOME/datasets/nuplan/exp"
replacing the path with the one you used.
the code expects $NUPLAN_DATA_ROOT
to contain the nuplan-v1.1
folder.
In a folder of your choice, run the following commands:
- Clone this project
git clone https://github.com/NVlabs/perp-fail-severity.git
- Clone and patch Trajectron++
git clone https://github.com/StanfordASL/Trajectron-plus-plus.git
cd Trajectron-plus-plus
git checkout 1031c7bd1a444273af378c1ec1dcca907ba59830
git apply ../perp-fail-severity/patches/trajectron.patch
cd ..
- Clone and patch NuPlan-devkit
git clone -b nuplan-devkit-v1.1 https://github.com/motional/nuplan-devkit.git
cd nuplan-devkit
git apply ../perp-fail-severity/patches/nuplan-v1.1.patch
cd ..
- Install all dependencies
cd perp-fail-severity
poetry install
This will set up the virtual environment with everything you need to run the code.
Note: by default this repository uses CPU-only JAX.
If you want to use the GPU version (which requires additional dependencies), comment L24 in pyproject.toml
and and uncomment L25.
Then run poetry install
or poetry update
if you installed the dependencies already.
Before being able to run the code you must precompute the HJ-Reachability lookup table. To do so, run
poetry run python3 precompute_value_function.py
The configuration is done through the configs/general.yaml
file.
To read more about configuration read the configuration guide.
To run the code, run
poetry run main
To visualize the scenario with nuBoard, run
poetry run viz experiments/2021-07-01_15-00-00 --nuboard
where experiments/2021-07-01_15-00-00
is the path to the experiment folder.
You get generate a report with
poetry run viz experiments/2021-07-01_15-00-00 --summary
or generate all plots (risk, timing, and confusion matrices) with
poetry run viz experiments/2021-07-01_15-00-00 --plot-all
This code is formatted with Black (automatically installed). You can format the code from your preferred IDE (e.g., VSCode), or, alternatively, you can run
poetry run poe format
to format all files in the project.
Please: before pushing to git, make sure the project is properly formatted.
This repository uses pre-commit to make sure the code satisfies basic best practices. To see/edit the hooks please read pre-commit config file. When you clone the repository, the hooks are not automatically enabled, to do so run
poetry run pre-commit install
from now on, all the hooks will automatically execute every time you try to push. If any of the hooks fail, the code will not be pushed.
If you want to manually run all pre-commit hooks on a repository, run poetry run pre-commit run --all-files
.
This will check that staged and committed files are compliant.
- To use nuBoard from a remote machine you might need to setup port forwarding. On your machine run
ssh -NfL localhost:5006:localhost:5006 <user>@<remote>
. - To omit debug log, run
export LOGURU_LEVEL=INFO
in the terminal session
If you use this code or the dataset, please cite the following paper:
@inproceedings{antonante23rss-perceptionFailureSeverity,
title = {Task-Aware Risk Estimation of Perception Failures for Autonomous Vehicles},
author = {P. Antonante, S. Veer, K. Leung, X. Weng, L. Carlone and M. Pavone},
booktitle = {Proceedings of Robotics: Science and Systems},
year = {2023},
}
The source code is released under the NSCL licence. The preprocessed dataset and pretrained models are under the CC BY-NC-SA 4.0 licence.