/stun

Implementation for the paper: STUN: Self-Teaching Uncertainty Estimation for Place Recognition

Primary LanguagePython

[IROS2022] STUN: Self-Teaching Uncertainty Estimation for Place Recognition

0. Environment Setup ⚙️

  • Ubuntu 18.04, python 3.8, A100
  • PyTorch 1.8.1 + CUDA 11.1

1. Pittsburgh dataset and pretrained models 📨

bash ./download.sh

then the files will be downloaded and saved in the following folders:

pittsburgh
├── database
├── query
├── structure

logs
├── student_contrast
├── student_quadruplet
├── student_triplet
├── teacher_triplet

2. Evaluate STUN 🔍

  1. STUN

    python main.py  --resume=logs/student_triplet/ckpt.pth.tar
  2. STUN (Constrast)

    python main.py --resume=logs/student_constrast/ckpt.pth.tar
  3. STUN (Quadruplet)

    python main.py --resume=logs/student_quadruplet/ckpt.pth.tar
  4. Standard Triplet

    python main.py --phase=test_tea	 --resume=logs/teacher_triplet/ckpt.pth.tar

3. Plot results 📈

python vis_results.py
# you can plot results of different models by populate the NETWORK variable.

ece.png

r@1/5/10 ↑ mAP@1/5/10 ↑ AP ↑ ECE{r@1/5/10}↓ ECE{mAP@1/5/10}↓ ECE{AP}↓
Standard Triplet 0.548 / 0.802 / 0.877 0.548 / 0.268 / 0.167 0.678 - - -
MC Dropout 0.302 / 0.523 / 0.611 0.302 / 0.108 / 0.061 0.463 0.205 / 0.075 / 0.112 0.205 / 0.396 / 0.443 0.111
PFE w/o MLS 0.550 / 0.805 / 0.876 0.550 / 0.266 / 0.167 0.690 0.413 / 0.160 / 0.092 0.413 / 0.695 / 0.795 0.275
PFE w/ MLS 0.444 / 0.680 / 0.764 0.444 / 0.199 / 0.120 0.655 0.517 / 0.284 / 0.200 0.517 / 0.762 / 0.841 0.336
BTL 0.515 / 0.766 / 0.840 0.515 / 0.252 / 0.158 0.591 0.236 / 0.058 / 0.105 0.236 / 0.497 / 0.591 0.131
Ours: STUN 0.613 / 0.840 / 0.898 0.613 / 0.280 / 0.171 0.739 0.171 / 0.084 / 0.127 0.171 / 0.491 / 0.600 0.067
Ours: STUN (Contrastive) 0.512 / 0.767 / 0.845 0.512 / 0.204 / 0.119 0.610 0.185 / 0.086 / 0.152 0.185 / 0.493 / 0.577 0.054
Ours: STUN (Quadruplet) 0.625 / 0.846 / 0.902 0.625 / 0.294 / 0.180 0.740 0.114 / 0.140 / 0.194 0.114 / 0.412 / 0.524 0.072