REMEMBERING HISTORY WITH CONVOLUTIONAL LSTM FOR ANOMALY DETECTION
By Weixin Luo$^{}$, Wen Liu$^{}$, Shenghua Gao
Introduction
All codes are based on Caffe. The main modifications are following
- Adding a convolutional lstm layer
- Changing the original Blob datastructure from 4-dimension tensor
$(N \times C \times H \times W)$ to 5-dimension tensor$(T \times N \times C \times H \times W)$ tensor. Be careful that do not use cuDNN engien in all activation function layers in this motidified Caffe version, such as ReLU, sigmoid, tanh and so on, because cuDNN do not support 5-dimension in these activation function (but Conv layer is ok). If do that, it will be wrong. - Adding a video data layers
Requirements for compiling
Since all the code are implemented on Caffe, so that in order to compile successfully, it must satisfy all requirements of Caffe (see the installization instruction). Whatmore, it also need to compile with Opencv 3.+, because the video data layer uses opencv libraries.
Datasets
- CUHK Avenue
- UCSD Pedestrians 1 & 2
- Subway Enter & Exit please to contact the orginal author to get the download links.
Training
As CUHK Avenue dataset for example, details are showd in the zstorm_conv_lstm_deconv/solver.prototxt and zstorm_conv_lstm_deconv/train.prototxt.
Testing or evaluation
Details are in the python shell, zstorm_conv_lstm_deconv/test.py.
License
All code are following the license of Caffe, and Caffe is released under the BSD 2-Clause license.
Citation
If you find this useful, please cite our work as follows:
@INPROCEEDINGS{luo2017remember,
author={W. Luo, W. Liu and S. Gao},
booktitle={2017 IEEE International Conference on Multimedia and Expo (ICME)},
title={Remembering history with convolutional LSTM for anomaly detection},
year={2017}
}