VP-Estimation-with-Prior-Gravity

Implementation of the paper Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction (ICCV 2023)

We propose a set of new solvers to extract the 3 orthogonal vanishing points of an uncalibrated image (i.e. unknown focal length), given a prior on the gravity direction. This repo assumes a Manhattan world with 3 main directions x, y, z. We also give the implementation of previous solvers that also work without prior gravity or with calibrated images.

Installation

Dependencies

The following dependencies need to be installed on your system:

GCC >= 9
CMake >= 3.17
Eigen >= 3.4
Ceres

Instructions

Clone the repository and its submodules: git clone --recurse-submodules https://github.com/cvg/VP-Estimation-with-Prior-Gravity.git

Install the necessary requirements and the Python package:

pip install -r requirements.txt
pip install -Ive .

Demo usage

Check out the demo notebook for an example of VP estimation with our hybrid solver.

Which solver shall I use?

Calibrated images	Known gravity	Solver to use
Yes	Yes	calibrated_100g
Yes	No	calibrated_210
No	Yes	hybrid
No	No	uncalibrated_220 or uncalibrated_211

Evaluation

Download the test datasets: YorkUrbanDB, ScanNet (follow the instruction in the NeurVPS ReadMe), and Lamar (we use the 4 sequences of the validation query images of the CAB building, located in CAB/sessions/query_val_hololens).

Then run one of the scripts runners/run_york_urban.py, runners/run_scannet.py, or runners/run_lamar.py. For example, a typical run on YorkUrban would be:

python runnners/run_york_urban.py <name of the solver> <path to the YorkUrban dataset> --nms non_orth --line_detector lsd --use_gt_gravity --magsac_scoring

Here are a few explanations on the options:

nms: use 'non_orth' to use our non minimal solver, or 'ceres' to use the same followed by a Ceres refinement (slower).
line_detector: 'lsd' (see https://github.com/iago-suarez/pytlsd) and 'deeplsd' (see https://github.com/cvg/DeepLSD) are currently supported. DeepLSD will probably produce better results but requires a GPU for fast inference.
use_gt_gravity: when using this flag, it will be using the ground truth gravity of the dataset (disabled by default).
magsac_scoring: when using this flag, RANSAC will use the MAGSAC scoring (disabled by default).

Bibtex

If you use this code in your project, please consider citing the following paper:

@InProceedings{Pautrat_2023_UncalibratedVP,
    author = {Pautrat, Rémi and Liu, Shaohui and Hruby, Petr and Pollefeys, Marc and Barath, Daniel},
    title = {Vanishing Point Estimation in Uncalibrated Images with Prior Gravity Direction},
    booktitle = {International Conference on Computer Vision (ICCV)},
    year = {2023},
}

xepost/VP-Estimation-with-Prior-Gravity