muralikrishnasn/SA-ConvLSTM-Pytorch

Pytorch implementation of Self-Attention ConvLSTM

Overview

• Pytorch implementation of Self-Attention ConvLSTM for Spatiotemporal Prediction.
• Pytorch implementation of ConvLSTM.

(testing on MovingMNIST)

Examples

ConvLSTM

python -m examples.moving_mnist_convlstm  

Self-Attention ConvLSTM

python -m examples.moving_mnist_self_attention_memory_convlstm

Directories

convlstm/

ConvLSTM implementation based on Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting.

self_attention_convlstm/

Self Attention ConvLSTM implementation based on Self-Attention ConvLSTM for Spatiotemporal Prediction.

self_attention_memory_convlstm/

Self-Attention ConvLSTM (with memory module) implementation based on Self-Attention ConvLSTM for Spatiotemporal Prediction.

Visualized Attention Maps

The above figure is SAM-ConvLSTM formulation process. alpha_{h} in the figure is used for visualizing attention maps in evaluation (pipeline/evaluator.py). Also see the following files for all calculation process.

• self_attention_memory_convlstm/cell.py
• self_attention_memory_convlstm/model.py
• self_attention_memory_convlstm/self_attention_memory.py