/DualCamSeg

Foreground extraction via dual-side cameras on a mobile device using long short-term trajectory analysis

Primary LanguageMATLABMIT LicenseMIT

Foreground extraction via dual-side cameras on a mobile device using long short-term trajectory analysis

If you are interested in using our dataset and find it useful in your research, please consider citing the following paper:

@article{lu2019foreground,
  title={Foreground extraction via dual-side cameras on a mobile device using long short-term trajectory analysis},
  author={Lu, Yucheng and Kim, Dong-Wook and Kim, Sesong and Jung, Seung-Won},
  journal={Image and Vision Computing},
  year={2019},
  publisher={Elsevier}
}

Third-party dependencies

This repository requires several third-party dependencies from various authors, which are all included in "thirdParty" folder. They are:

  • Large displacement optical flow from Thomas Brox et. al.;

  • Dense point trajectories from N. Sundaram et. al.;

  • Optical flow visualization untils from Daniel Scharstein;

  • Color chart visualization samples from Tim Holy;

  • Bilateral space video segmentation from Nicolas Maerki et. al.

Dataset details

The dataset used in the paper can be downloaded from Google drive here. Please check "motionType.txt" for descriptions of each sample clip.

How to use

Simply download the dataset and unzip it to "dataset" folder, or use your own video clips. Then run "DEMO.m" to perform foreground segmentation. If the input is a video, it will first separate dual-camera images and save them as PNG files. Please note that the pictures of some devices are mirrored and thus need to be flipped back again.

Below is an example output of the combination of initial labeling result, semi-dense classification, and final dense segmentations:

demo