VideoPopup

This is the implementation of the following ECCV2014 paper:

Video Pop-up: Monocular 3D Reconstruction of Dynamic Scenes
Chris Russell*, Rui Yu*, Lourdes Agapito
*Joint first authorship

and the reconstruction part of:

Dense Monocular Depth Estimation in Complex Dynamic Scenes
Rene Ranftl, Vibhav Vineet, Qifeng Chen, Vladlen Koltun

For more information about this work, please visit the project website.

This github repository is maintained by Rui Yu (R.Yu@cs.ucl.ac.uk). Contact me if you have any questions.

1. Building the System

VideoPopup has been tested in Ubuntu 14.04 only.

1.1 Requirements

Vispy: we use vispy for 3D visualization

CVXPY: if you want to try depth reconstruction

OpenSfM: see this Dockerfile for the commands to install all dependencies for OpenSfM.

Our code base already contains a modified version of OpenSfM, so you do not need to install OpenSfM yourself.

1.2 Build VideoPopup

To compile the system, do the following:

  ./build.sh

2. Data

One example sequence is available at google drive.

put data folder under root directory
if you want to create new tracks file, check data/Kitti/05_2f/runBroxMalik.sh

3. Examples

After building the code and downloading the data, you are ready to try the provided examples.

3.1 Motion Segmentation

video_popup/motion_segmentation/video_popup_motseg.py
video_popup/motion_segmentation/segmentation_check.py

3.2 Depth Reconstruction from two consecutive frames

video_popup/depth_reconstruction/depth_reconstruction_test.py

3.3 Piecewise Orthographic Reconstruction

video_popup/reconstruction/reconstruction_bird.py
video_popup/reconstruction/reconstruction_bird_check.py

3.4 Piecewise Perspective Reconstruction

video_popup/reconstruction/reconstruction_tm_kitti.py
video_popup/reconstruction/reconstruction_tm_kitti_check.py

4. GUI Usage

Keyboard interactions

  'q': save rendering image
  'c': switch between image color and segmentation color
  's': decrease point size
  'b': increase point size
  'Right': next frame
  'Left: previous frame

If you use vispy_viewer.py, you can also do

  ' ': reset to original viewpoint
  'p': show pointcloud
  'f': show mesh faces

cvfish/VideoPopup