/streaming-vis-pca

Incremental PCA for Visualizing Streaming Multidimensional Data

Primary LanguageJupyter NotebookBSD 3-Clause "New" or "Revised" LicenseBSD-3-Clause

C++ Library and Python Module for Incremental PCA for Streaming Multidimensional Data

Incremental PCA for visualizing streaming multidimensional data from [Fujiwara et al., 2019].

New

  • Supported installation using virtual environments (due to the use of venv became a default from Python3.12)
  • All major OSs, Mac OS, Linux, Windows, are supported now (2022-04-28).

About

  • Incremental PCA for visualizing streaming multidimensional data from:
    An Incremental Dimensionality Reduction Method for Visualizing Streaming Multidimensional Data
    Takanori Fujiwara, Jia-Kai Chou, Shilpika, Panpan Xu, Liu Ren, and Kwan-Liu Ma
    IEEE Transactions on Visualization and Computer Graphics and IEEE VIS 2020 (InfoVis 2019). DOI: 10.1109/TVCG.2019.2934433

  • Demonstration videos from the application implemented in the paper above are available here. Also, from the same site, you can download a source code to reproduce the performance evaluation in the paper.

  • Features

    • Fast C++ implementation with Eigen3 of Incremental PCA from [Ross et al., 2008].

      • D. Ross, J. Lim, R. Lin, M. Yang. Incremental Learning for Robust Visual Tracking, International Journal of Computer Vision, Volume 77, Issue 1-3, pp. 125-141, 2008.

    • Mental map preservation with Procrustes transformation.

    • Position estimation for handling new data points' incomplete features.

    • Uncertainty measures for the position estimation.

Requirements

  • C++11 compiler, Python3, Eigen3, Pybind11, Numpy

  • Note: Tested on macOS Sonoma, Ubuntu 22.0.4 LTS, Windows 10.

Setup

Mac OS with Homebrew

  • Make sure if you have C++ compiler. For example,

    which c++

    should return the c++ compiler path (e.g., /usr/bin/c++) if it exists. If it does not exist, run:

    xcode-select --install

  • Download and move to this repository. Then run presetup.py

    python3 presetup.py

    • Note (2024-04-14): For installation using virtualevn, proccesses listed in "presetup.py" are separated from "setup.py". This is due to venv's bug (module involved commands do not work correctly when using pip3 under venv).
    • This python script generates C++ shared library of inc-pca also installs homebrew, pkg-config, python3, eigen, pybind11 if they do not exist.

  • Install the modules with pip3 (this also installs homebrew, pkg-config, python3, eigen, pybind11, numpy, if they do not exist).

    pip3 install .

Linux (tested on Ubuntu 22.0.4 LTS)

  • Install libraries

    sudo apt update

    sudo apt install libeigen3-dev python3-pip python3-dev

    • Note: Replace apt commands based on your Linux OS.

  • Install the modules with pip3.

    python3 presetup.py

    then,

    pip3 install .

    • Note: If installation does not work, check setup.py and replace c++ commands based on your environment.

Windows (tested on Windows 10, requiring MSVC as a C++ compiler)

  • Install required compiler and library

  • Download and move to this repository. Then run presetup.py

  • Install the modules with pip3 in "Command Prompt for VS". Note: if you installed 64-bit Python3, use x64 Native Command Prompt for VS.

    python3 presetup.py

    then,

    pip3 install .

Usage

  • With Python3

    • Import "inc_pca" from python (from inc_pca import IncPCA). See sample.ipynb (jupyter notebook), inc_pca.py, or docs/index.html for detailed usage examples.

  • With C++

    • Include inc_pca.hpp in C++ code with inc_pca.cpp.

How to Cite

Please, cite:
Takanori Fujiwara, Jia-Kai Chou, Shilpika, Panpan Xu, Liu Ren, and Kwan-Liu Ma, "An Incremental Dimensionality Reduction Method for Visualizing Streaming Multidimensional Data". IEEE Transactions on Visualization and Computer Graphics, Vol. 26, No. 1, pp. 418-428, 2020. DOI: 10.1109/TVCG.2019.2934433