Official Implementation of our paper Learned Fourier Bases for Deep Set Feature Extractors in Automotive Reinforcement Learning. You can find the draft here, or the final paper here.
Neural networks in the automotive sector commonly have to process varying number of objects per observation. Deep Set feature extractors have shown great success on problems in reinforcement learning with dynamic observations, achieving state-of-the-art performance on common tasks like highway driving. However, recent work has shown that neural networks suffer from a spectral bias, fitting to low frequencies of scalar features used in such representations, like velocities and distances. We introduce a novel set feature extractor combining learned Fourier features with the Deep Set architecture. The proposed architecture reduces this spectral bias allowing more sample-efficient training and better performance by learning a more detailed representation. Extensive experiments are conducted on three different environments, including two novel environments featuring a large number of objects in challenging random scenarios. Our method outperforms state-of-the-art approaches and exceeds the performance of existing Deep Set architectures on these challenging new tasks.
Navigating a tightly winding road with slippery surface using a dynamic vehicle model:
Parking into a spot parallel to the road using a kinematic vehicle model:
Overtaking vehicles on the highway-env environment:
We recommend installing to a new conda environment:
conda create -n lffds python=3.8 pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch
conda activate lffds
pip install -e CarEnv
pip install -r requirements.txtExecute train.py to repeat experiments from our paper.
The following parameters are available to configure training including the competitors used:
| Argument | Description |
|---|---|
problem |
Either country, parking or highway, depending on the problem to train on. |
extractor |
One of cnn, flat, deepset, lffds to select an architecture from the paper, where lffds is ours. |
gamma |
Discount factor |
steps |
Gradient steps, defaults to value from paper. |
eval_frequency |
Gradient steps between evaluation, defaults to value from paper. |
Please note that we have used W&B for logging, so you may require an account to log output without modifications to the code.
Please cite this work as:
@inproceedings { SchRei2023b,
author = {Schier, Maximilian and Reinders, Christoph and Rosenhahn, Bodo},
title = {Learned Fourier Bases for Deep Set Feature Extractors in Automotive Reinforcement Learning},
booktitle = {2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC)},
year = {2023},
pages = {931-938},
doi = {10.1109/ITSC57777.2023.10422233}
}