XiaohuLuVPDetection

This is a Python + OpenCV implementation of the Vanishing Point algorithm by Xiaohu Lu et al. - http://xiaohulugo.github.io/papers/Vanishing_Point_Detection_WACV2017.pdf

Requirements

Python 3
OpenCV Contrib 3.x or 4.0.0.21 - Please note that any version higher than 4.0.0.21 does not have the LSD detection algorithm included, which is a key component for this method to work. If you have a higher version of OpenCV installed, this will not work.
NumPy

Setup

Method #1 - Cloning this repo and installing locally

Simply clone this repo, then run the included setup.py file to get it to install onto your local machine:

$ python setup.py build
$ python setup.py install

Method #2 - Through PyPI

This project is now available through PyPI: https://pypi.org/project/lu-vp-detect/

If don't want to clone this repo, you can simply use pip install to install this project on your system.

$ pip install lu-vp-detect

After installation

The lu_vp_detect package should be installed at this point which will contain the implementation of the algorithm as well as a command-line test script to test out the algorithm.

How to use

The main detection algorithm is written in the lu_vp_detect package and is implemented in the file vp_detection.py. The VPDetection class is what is required and there are two methods of interest in the class:

find_vps: Finds the vanishing points in normalized 3D space
create_debug_VP_image: Creates a debug image for showing which detected lines in the image align with each vanishing point.

The main parameters for affecting performance are:

length_thresh: The minimum length of the lines detected to find the vanishing points
principal_point: The principal point of the camera that took the image (in pixels). The default is to assume the image centre.
focal_length: The focal length of the camera (in pixels). The default is 1500.
seed: An optional integer ID that specifies the seed for reproducibility as part of the algorithm uses RANSAC. Default is None so no seed is used.

Simply create a VPDetection object with the desired parameters and run the detection algorithm with the desired image. You can read in the image yourself or you can provide a path to the image. Note that the returned vanishing points will be a 3 x 3 NumPy array such that the first row corresponds to the vanishing point appearing to the right of the image, the second row corresponds to the vanishing point appearing to the left of the image and the last row corresponding to the vertical vanishing point:

from lu_vp_detect import VPDetection
length_thresh = ... # Minimum length of the line in pixels
principal_point = (...,...) # Specify a list or tuple of two coordinates
                            # First value is the x or column coordinate
                            # Second value is the y or row coordinate
focal_length = ... # Specify focal length in pixels
seed = None # Or specify whatever ID you want (integer)

vpd = VPDetection(length_thresh, principal_point, focal_length, seed)

img = '...' # Provide a path to the image
# or you can read in the image yourself
# img = cv2.imread(path_to_image, -1)

# Run detection
vps = vpd.find_vps(img)

# Display vanishing points
print(vps)

# You can also access the VPs directly as a property
# Only works when you run the algorithm first
# vps = vpd.vps

# You can also access the image coordinates for the vanishing points
# Only works when you run the algorithm first
# vps_2D = vpd.vps_2D

You can optionally create a debug image that shows which detected lines align with which vanishing point in the image. They are colour coded so that each unique colour corresponds to the lines that provide support for a vanishing point. Lines that are black correspond to "outlier" lines, meaning that they did not contribute any information in calculating any vanishing points.

# Create debug image
vpd.create_debug_VP_image(show_image=True, save_image='./path/to/debug.png')

The show_image flag will open a OpenCV imshow window that will display the image. The save_image if it's not set to None will write the corresponding image to file. Of course, different combinations of the input variables will allow you to customize what outputs you want to consume or save.

Command-line test script

The run_vp_detect.py file is a command-line script that uses argparse to take in the parameters from the command-line, runs the algorithm and shows the vanishing points in both 3D normalised space and 2D image coordinate space in the console. You can additionally show the debug image and save the debug image to file by setting the right parameters.

$ run_vp_detect -h
usage: run_vp_detect [-h] -i IMAGE_PATH [-lt LENGTH_THRESH]
                     [-pp PRINCIPAL_POINT PRINCIPAL_POINT] [-f FOCAL_LENGTH]
                     [-d] [-ds] [-dp DEBUG_PATH] [-s SEED]

Main script for Lu's Vanishing Point Algorithm

optional arguments:
  -h, --help            show this help message and exit
  -i IMAGE_PATH, --image-path IMAGE_PATH
                        Path to the input image
  -lt LENGTH_THRESH, --length-thresh LENGTH_THRESH
                        Minimum line length (in pixels) for detecting lines
  -pp PRINCIPAL_POINT PRINCIPAL_POINT, --principal-point PRINCIPAL_POINT PRINCIPAL_POINT
                        Principal point of the camera (default is image
                        centre)
  -f FOCAL_LENGTH, --focal-length FOCAL_LENGTH
                        Focal length of the camera (in pixels)
  -d, --debug           Turn on debug image mode
  -ds, --debug-show     Show the debug image in an OpenCV window
  -dp DEBUG_PATH, --debug-path DEBUG_PATH
                        Path for writing the debug image
  -s SEED, --seed SEED  Specify random seed for reproducible results

Take note that the principal point requires two arguments so you can separate the x and y coordinate by a space.

Example running of the script

A sample image was extracted from TinyHomes.ie, specifically this image: https://www.tinyhomes.ie/wp-content/uploads/2015/09/02.jpg. It is saved as test_image.jpg in this repository.

The image has been reduced down to 1/4 of the resolution for ease. The new image has dimensions of 968 x 648. The image was originally taken with a Nikon D40X whose focal length is 18 mm and the CCD width is 15.8 mm. Therefore, the estimated focal length should be:

968 x (18 / 15.8) = 1102.79 pixels

To test the algorithm and reproduce the results below, set the seed to be 1337 and set the minimum line length to be 60 pixels. Also, set the focal length to be what was computed above. The principal point is assumed to be the centre of the image.

$ run_vp_detect -i ./test_image.jpg -f 1102.79 -s 1337 -ds -lt 60

For completeness, to do this programmatically:

from lu_vp_detect import VPDetection
length_thresh = 60
principal_point = None
focal_length = 1102.79
seed = 1337

img = './test_image.jpg'

vpd = VPDetection(length_thresh, principal_point, focal_length, seed)
vps = vpd.find_vps(img)
print(vps)

We get the following output and debug image:

Input path: ./test_image.jpg
Seed: 1337
Line length threshold: 60.0
Focal length: 1102.79
Principal point: [484. 324.]
The vanishing points in 3D space are:
Vanishing Point 1: [0.3774699  0.01993015 0.92580736]
Vanishing Point 2: [-0.9260219   0.00812403  0.37738246]
Vanishing Point 3: [-0.         -0.9997684   0.02152233]

The vanishing points in image coordinates are:
Vanishing Point 1: [933.6292 347.7401]
Vanishing Point 2: [-2222.0286   347.7401]
Vanishing Point 3: [   484.    -50903.473]
Creating debug image and showing to the screen

License

This code is currently distributed under the MIT license. Please modify and use the code as you see fit. I only ask that you not only include this license as part of your work but to please acknowledge where you got this work from!

Xiaoyang-Rebecca/XiaohuLuVPDetection