Patrik Vacek1
David Hurych2
Karel Zimmermann1
Tomas Svoboda1
1CTU Deparment of Cybernetics
2Valeo AI
- 🚀 Scalability:
Let It Flow
does not require training or supervision as an optimization-based method and works on out-of-distribution data. - 🌈 Real Scene Flow:
Let It Flow
performs consistently accross real world benchmark datasets and not only on StereoKITTI. Improvements are observable even on new dynamic/object class error metrics. - ⚖️ Generalizability:
Let It Flow
does not have to adjust the parameter config for accurate results on the tested datasets.
Tested with Python==3.10, PyTorch3d, PyTorch Scatter See install.sh for installation of libraries or run it directly:
conda create -n let-it-flow python=3.10
conda activate let-it-flow
conda install pytorch=1.13.0 torchvision pytorch-cuda=11.6 -c pytorch -c nvidia
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
git clone https://github.com/facebookresearch/pytorch3d.git
cd pytorch3d && python3 -m pip install -e .
cd ..
conda install pytorch-scatter -c pyg
Download and unpack the Argoverse2 preprocessed sequence samples into the dataset/argoverse2/
path.
tar -xvf argoverse2.tgz datasets/argoverse2/
Set up the config.yaml to include path to the argoverse2 folder
and path to store the results
.
To infer the model on Argoverse2 dataset sequences, please run the following:
python run_optimization.py
The script will store the outputs (and inputs with Ground truth) into the store_path
for metric calculation.
After the model inference (takes about 2 hours on GPU), generate metrics of Let It Flow (lif)
by running:
python compute_metrics.py lif
Result will print the End-Point-Error per class annotated in Argoverse2 and also overall ThreewayEPE.
If you find our work helpful, please consider citing our paper:
@misc{vacek2024lif,
title={Let It Flow: Simultaneous Optimization of 3D Flow and Object Clustering},
author={Patrik Vacek and David Hurych and Tomáš Svoboda and Karel Zimmermann},
year={2024},
eprint={2404.08363},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
This work is under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This work is developed based on the open-source repositories and baselines: KISS-ICP, NSFP, MBNSFP, Chodosh
❤️ We thank to the contributors from the above open-source repositories.