/ConvLSTM-Pytorch

Primary LanguagePythonMIT LicenseMIT

Notice

This repo is heavily borrowed from here and this repo will be modified to meet my personal demands. Please clone the original repo instead. All the following informations from from the README.md in the original repo. Thanks!

To run it with CloudCast dataset:
With ConvLSTM:

python main_cloudcast.py -clstm

With ConvGRU:

python main_cloudcast.py -cgru

ConvLSTM-Pytorch

ConvRNN cell

Implement ConvLSTM/ConvGRU cell with Pytorch. This idea has been proposed in this paper: Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting

Experiments with ConvLSTM on MovingMNIST

Encoder-decoder structure. Takes in a sequence of 10 movingMNIST fames and attempts to output the remaining frames.

Instructions

Requires Pytorch v1.1 or later (and GPUs)

Clone repository

git clone https://github.com/jhhuang96/ConvLSTM-PyTorch.git

To run endoder-decoder network for prediction moving-mnist:

python main.py

Moving Mnist Generator

The script data/mm.py is the script to generate customized Moving Mnist based on MNIST.

MovingMNIST(is_train=True,
            root='data/',
            n_frames_input=args.frames_input,
            n_frames_output=args.frames_output,
            num_objects=[3])
  • is_train: If True, use script to generate data. If False, directly use Moving Mnist data downloaded from http://www.cs.toronto.edu/~nitish/unsupervised_video/
  • root: The path of MNIST data
  • n_frames_input: Number of input frames (int)
  • n_frames_output: Number of output frames (int)
  • num_objects: Number of digits in a frame (List) . [3] means there are 3 digits in each frame

Result

Result

  • The first line is the real data for the first 10 frames
  • The second line is prediction of the model for the last 10 frames

Citation

@inproceedings{xingjian2015convolutional,
  title={Convolutional LSTM network: A machine learning approach for precipitation nowcasting},
  author={Xingjian, SHI and Chen, Zhourong and Wang, Hao and Yeung, Dit-Yan and Wong, Wai-Kin and Woo, Wang-chun},
  booktitle={Advances in neural information processing systems},
  pages={802--810},
  year={2015}
}
@inproceedings{xingjian2017deep,
    title={Deep learning for precipitation nowcasting: a benchmark and a new model},
    author={Shi, Xingjian and Gao, Zhihan and Lausen, Leonard and Wang, Hao and Yeung, Dit-Yan and Wong, Wai-kin and Woo, Wang-chun},
    booktitle={Advances in Neural Information Processing Systems},
    year={2017}
}