Panoramic Robust PCA

This package contains the implementation and example usages of the Panoramic Robust Principal Component Analysis (PRPCA) method, an algorithm for producing robust estimates of the foreground and background of a scene from possibly corrupted video with arbitrary camera motion. For more information, see the project website.

If you have any questions, contact Chen Gao.

Quickstart

To generate some reconstructions using PRPCA, simply run the examples in demo.m. The script contains examples of processing both grayscale and color videos in the noiseless and noisy (salt and pepper) regimes.

Data

data/tennis: 35 frames of the Tennis sequence. Color images with resolution 480 x 854. Also contains homographies computed by the registerVideo function for mapping each frame to the coordinates of the 17th frame of the sequence

Scripts

demo.m: Example uses of PRPCA to perform robust foreground-background separation from video with arbitrary camera motion

Main Functions

src/PRPCA.m: Main function for the PRCPA method as described in the papers cited below
src/PRPCA_noiseless.m: An implementation of PRPCA intended for the noiseless (possibly moving camera) setting. This implementation uses ell-1 based regularization for the foreground and omits the total variation-regularized component, which is not essential when there is no corruption to disentangle from the foreground component

Other Functions

src/adjustLS.m: Adjusts the low-rank and sparse components of a total variation-regularized foreground-background decomposition to account for the constant-offset-invariance of the TV penalty
src/augRobustPCA.m: Solves the Augmented Robust PCA problem with low-rank background, total variation-regularized foreground, and an addition sparse component to capture residual corruptions
src/cleanBackground.m: Cleans the background component of a panoramic foreground-background grayscale video reconstruction
src/cleanBackground_RGB.m: Cleans the background component of a panoramic foreground-background RGB video reconstruction
src/correctLighting.m: Attempts to correct non-uniform lighting effects in a panoramic image
src/formatForDisplay.m: Formats a decomposition for display by appropriately scaling and clamping the components to [0, 1]
src/OptShrink.m: Implementation of the data-driven OptShrink low-rank matrix estimator
src/pano2video.m: Transforms a panoramic grayscale video back to its native perspective
src/pano2video_RGB.m: Transforms a panoramic RGB video back to its native perspective
src/PlayMovie.m: Utility function for visualizing a data tensor as a video
src/registerVideo.m: registers the frames of a video to a common reference perspective
src/robustPCA.m: Approximately solves the Robust PCA problem using an OptShrink-based low-rank update
src/tvdn.m: Solves the subsampled total-variation with denoising (TVDN) problem
src/writeGIF.m: Utility function for writing a GIF image.

License

This code is made available with the MIT license. See LICENSE for details.

References

If you use this code, we request that you cite the following papers:

@article{moore2017panoramic,
    author    = {Moore, Brian E* and Gao, Chen* and Nadakuditi, Raj Rao},
    title     = {Panoramic Robust PCA for Foreground-Background Separation on Noisy, Free-Motion Camera Video},
    journal   = {IEEE Transactions on Computational Imaging},
    volume    = {5},
    number    = {2},
    pages     = {195--211},
    year      = {2019},
}

@inproceedings{gao2017augmented,
    author    = {Gao, Chen and Moore, Brian E and Nadakuditi, Raj Rao},
    title     = {Augmented robust PCA for foreground-background separation on noisy, moving camera video},
    booktitle = {2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)}
    month     = {November},
    year      = {2017},
    pages     = {1240-1244}
}

gaochen315/panoramicRPCA