/LSTM-Action-Recognition

Part of CSE 252C

Primary LanguagePython

This is the LSTM part of work for CSE 252C Project

Abstract

This repo implements Nitish and Elman's work Unsupervised Learning of Video Representations using LSTMs using Tensorflow framework.
The work includes

  • LSTM AutoEncoder
  • LSTM future frames prediction

Files

testEncoder.py provides single number autoEncode/Decode process. Simply run python testEncoder.py will get the results. update.py provides this process on UCF101 dataset, the input array could be

  • patches: 112x112x3 images directly
  • perceptons: 4096 dimensional fc6 features extracted from C3D model

extract-fc6.py provides the code to extract UCF101 fc6 features into bin files.

Dataset and Reference

  • UCF-101: Action Recognition Data Set
  • Extract Frames: Thanks to This Repo. Each single avi file is decoded with 5FPS in a single directory.
  • Model: The pre-trained C3D model for extracting fc6 features is from This work
  • Other Reference: Here is another raw implementation by Emansim