/two-stream-action-cnn-analysis

:mortar_board: Undergraduate thesis on analysing two stream CNNs for action recognition using excitation backpropagation

Primary LanguageJupyter Notebook

2SCNN Feature analysis

Working repository for feature analysis of 2SCNN

Models

Dependencies

  • Python 3
    • matplotlib
    • numpy
    • scipy
    • pandas
    • seaborn
    • click
    • skimage
  • Jupyter notebook (6.0)
  • IPykernel for Jupyter (5.0)
  • Bash
  • Caffe (Excitation BP fork), preferably built with GPU support.

Setup

  • Edit lib/config.py to point to your local excitation bp caffe installation, nets path (e.g. caffe/models, I like to keep mine in my home directory), and data root.

Repository overview

  • lib: Most of the meaty code lives in here, code for handling datasets, performing contrastive EBP, mapping from class ids to class names.
  • net_configs: Python library of configuration scripts defining settings peculiar to models.
  • scripts: Python CLI scripts for generating attention maps, stitching videos, graphing smoothness
  • generated: A few bash scripts live in here to call the python scripts that do most of the work.
  • notebooks: All the Jupyter notebooks used for experimental analysis reside here. They're meant to be run in numerical order. Notebooks with the same numerical prefix can be run in any order.
  • ./run_jupyter.sh is a helper script to setup my environment with the correct LD_LIBRARY_PATH and PATH env variables necessary to find Caffe, CuDNN etc.

Acknowledgments

The code for performing contrastive excitation backprop is from https://github.com/jimmie33/Caffe-ExcitationBP, I also took inspiration from the code used to overlay attention maps on images.