/DYAN

A Dynamical Atoms-Based Network For Video Prediction

Primary LanguagePython

DYAN

Overview

This repository provides training and testing code and data for ECCV 2018 paper:

"DYAN - A Dynamical Atoms-Based Network For Video Prediction", Wenqian Liu, Abhishek Sharma, Octavia Camps, and Mario Sznaier

Further information please contact Wenqian Liu at liu.wenqi@husky.neu.edu, Abhishek Sharma at sharma.abhis@husky.neu.edu.

Requirements

  • PyTorch NOTE: previous versions(0.3 or below) might not work!
  • Python 2.7
  • Cuda 9.0

Data Preparation

Getting started

  • set training/testing data directory:
rootDir = 'your own data directory'
  • Run the training script:
python train.py
  • Run the testing script: (need to set correct 'flowDir' in Test.py to your own optical flow files)
python Test.py

Evaluation

We adopts test and evaluation script for ICLR 2016 paper: "Deep multi-scale video prediction beyond mean square error", Michael Mathieu, Camille Couprie, Yann LeCun.

BeyondMSE paper code

  • Follow BeyondMSE's prerequisite to set up enviroment.
  • Include util/TestScript.lua from DYAN's folder into BeyondMSE's folder.
  • Set necessary directory and run:
th TestScript.lua

Generate Optical Flow Files

We adopt PyFlow pipeline to generate our OF files.

Clone pyflow repo and compile on your own machine. Then run our util/saveflows.py .

License and Citation

The use of this software is RESTRICTED to non-commercial research and educational purposes.

@InProceedings{Liu_2018_ECCV,
author = {Liu, Wenqian and Sharma, Abhishek and Camps, Octavia and Sznaier, Mario},
title = {DYAN: A Dynamical Atoms-Based Network For Video Prediction},
booktitle = {The European Conference on Computer Vision (ECCV)},
month = {September},
year = {2018}
}