This is the implementation of our ICRA 2021 " Robust 360-8PA": Redesigning The Normalized 8-point Algorithm for 360-FoV Images" (website project).
For a quick introduction (3 min), please here
June 3, 2021 - Code release
Sep 12, 2021: Dataset release
This REPO is our own implementation, in python, for a camera pose estimation using the eight-point algorithm [1], the non-linear optimization over residual errors (Gold Standard Method [GSM]) [2], and our method named Robust 360-8PA.
Using this implementation, you can:
- Track key-features, using LKT-tracker, from 360-FoV and Fish-eye images (from our MP3D-VO and TUM-VI[3] datasets, respectively).
- Sample 3D points from GT 360-depth maps (only for our MP3D-VO dataset), adding noise vMF, and outliers.
- Evaluate camera pose, with and without RANSAC, using 8-PA[1], GSM[2], and our Robust 360-8PA.
Further capabilities, analysis and resources are released in the branch dev
.
- python 3.7.7
- vispy 0.5.3
- numpy 1.18.5
- opencv-python 3.4.3.18
- pandas 1.0.5
- levmar 0.2.3
For convenience, our dataset MP3D_VO and TUM_VI [3] will be provided after filling out the following form and agreement VSLAB-Form
For convience, we implement a < Class config > to load the used settings in this repo, from a yaml file. e.g .config/config_TUM_VI.yaml
. You can use the following lines for loading this configuration.
from config import Cfg
config_file = Cfg.FILE_CONFIG_MP3D_VO # PATH to yaml file
cfg = Cfg.from_cfg_file(yaml_config=config_file)
the cfg
instance is used to set all of the classes and methods in this implementation. e.g.,
# from test/test_tracking_features.py
config_file = Cfg.FILE_CONFIG_TUM_VI
cfg = Cfg.from_cfg_file(yaml_config=config_file)
tracker = FeatureTracker(cfg)
# from test/test_saving_sampled_bearings.py
config_file = Cfg.FILE_CONFIG_MP3D_VO
cfg = Cfg.from_cfg_file(yaml_config=config_file)
sampler = BearingsSampler(cfg)
# test/test_eval_methods.py
config_file = Cfg.FILE_CONFIG_MP3D_VO
cfg = Cfg.from_cfg_file(yaml_config=config_file)
eval_solvers(cfg)
NOTE: In general cfg
is created at the beginning of every script in this implementation, e.g. in plots/plot_cam_pose_errors.py
if __name__ == '__main__':
config_file = Cfg.FILE_CONFIG_MP3D_VO
cfg = Cfg.from_cfg_file(yaml_config=config_file)
plot_sampling_evaluations(cfg)
There are three main ENV variables that have to be modified in env
file.
DIR_DATASETS=/HD/datasets #path to root dir of datasets
MP3D_VO_DATASET=${DIR_DATASETS}/ICRA2021 # path to MP3D-VO dataset
TUM_VI_DATASET=${DIR_DATASETS}/TUM_VI # path to TUM-VI dataset
After this. You should source the setup.sh
to export these variables and set the current implementation into your PYTHONPATH. This can be added to your .bashrc file if your are using LINUX.
# from root of this REPO
source setup.bash
To run this implementation, we present several run test files in /test
:
/test/test_tracking_features.py
/test/test_saving_tracked_features.py
/test/test_saving_sampled_bearings.py
/test/test_read_saved_bearings.py
/test/test_eval_methods.py
/test/test_eval_ransac_methods.py
In order to run one of these files, you just need to execute them as normal python script. e.g.
python test/test_eval_ransac_methods.py
output
Number of evaluated bearings: 400
Rot-e:6.135843e-02 Tran-e:1.302335e-01 ransac_8pa
Rot-e:5.519852e-02 Tran-e:8.634036e-02 ransac_opt_SK
Rot-e:6.127112e-02 Tran-e:9.963809e-02 ransac_GSM
Rot-e:3.659004e-02 Tran-e:8.599553e-02 ransac_GSM_const_wRT
Rot-e:2.893354e-02 Tran-e:5.370007e-02 ransac_GSM_const_wSK
To plot the MAE evaluations (median absolute errors), you can run:
python plots/plot_cam_pose_errors.py
- Credit of this repo is shared with Chin-Hsuan Wu.
Please cite our paper for any purpose of usage.
@INPROCEEDINGS{9560888,
author={Solarte, Bolivar and Wu, Chin-Hsuan and Lu, Kuan-Wei and Tsai, Yi-Hsuan and Chiu, Wei-Chen and Sun, Min},
booktitle={2021 IEEE International Conference on Robotics and Automation (ICRA)},
title={Robust 360-8PA: Redesigning The Normalized 8-point Algorithm for 360-FoV Images},
year={2021},
volume={},
number={},
pages={11032-11038},
doi={10.1109/ICRA48506.2021.9560888}}