amandajshao/Slicing-CNN

The source code for our CVPR 2016 work "Slicing Convolutional Neural Network for Crowd Video Understanding".

Slicing Convolutional Neural Network for Crowd Video Understanding

This is the source code for "Slicing Convolutional Neural Network for Crowd Video Understanding". It aims at learning generic spatio-temporal features from crowd videos, especially for long-term temporal learning (i.e. 100 frames).

Overview

Three-branch Slicing CNN model (i.e. xy-, xt-, and yt-branch)

Crowd attribute recognition (i.e. 94 crowd-related attributes)

Project Site

Caffe

A fork of the well-known Caffe framework with Multi-GPU training and Dimension Swap layer.

Apart from the official installation prerequisites, we have several other dependencies:

1. Install openmpi to allow multi-gpu running
2. Python packages (e.g. numpy, scipy, scikit-image, etc.)
3. Add export PYTHONPATH="[path_python_layer]:$PYTHONPATH" to ~/.bashrc and restart the terminal. Here [path_python_layer] indicates the absolute path of the python script of py_dim_swap_layer.py.

Get the Caffe code

git clone --recursive https://github.com/amandajshao/Slicing-CNN.git

Files

• Dataset

  The dataset is introduced in CVPR 2015 which contains 10,000 crowd videos from 8,257 different crowded scenes with annotated 94 attributes.

• LMDB Data

  The LMDB data used in the model with training/validation/test splits.

• CNN Initial Model

  The initial model (VGG-16) is pre-trained on UCF-101 action dataset (single frame) and fine-tuned on WWW dataset (single frame).

  Dropbox link

  BaiduDisk link: http://pan.baidu.com/s/1jH5VLNw (password: 76zl)

• CNN Best Model

  Three models: SCNN-xy, SCNN-xt, SCNN-yt.

  Dropbox link

  BaiduDisk link: http://pan.baidu.com/s/1pK7h5sJ (password: j024)

• Prototxt

  The prototxts are corresponding to the above three models (SCNN-xy/-xt/-yt).

  Dropbox link

  BaiduDisk link: http://pan.baidu.com/s/1o85xUI2 (password: mwvo)

• Scripts

  There are two scripts provided in our code: model_run.sh and extract_features.sh.

Related Projects

Deeply Learned Attributes for Crowd Scene Understanding

Thanks

• Kai Kang
• Tong Xiao
• Yuanjun Xiong

Citation

J. Shao, C. C. Loy, K. Kang, and X. Wang. Slicing Convolutional Neural Network for Crowd Video Understanding. Computer Vision and Pattern Recognition (CVPR), 2016.

@article{shao2016scnn,
  title={Slicing Convolutional Neural Network for Crowd Video Understanding},
  author={Shao, Jing and Loy, Chen Change and Kang, Kai and Wang, Xiaogang},
  booktitle={Computer Vision and Pattern Recognition (CVPR)},
  year={2016}
}