for Joint Optical Flow and Occlusion Estimation
Iterative Residual Refinement This repository is the PyTorch implementation of the paper:
Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation (CVPR 2019)
Junhwa Hur and Stefan Roth
Department of Computer Science, TU Darmstadt
[Preprint] [Proceeding] [Supplemental]
Please cite the paper below if you find our paper and source codes are useful.
@inproceedings{Hur:2019:IRR,
Author = {Junhwa Hur and Stefan Roth},
Booktitle = {CVPR},
Title = {Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation},
Year = {2019}
}
Contact: Junhwa Hur [fname.lname]@visinf.tu-darmstadt.de
Getting started
This code has been developed under Anaconda(Python 3.6), Pytorch 0.4.1 and CUDA 8.0 on Ubuntu 16.04.
-
Please install the followings:
- Anaconda (Python 3.6)
- PyTorch 0.4.1 (Linux, Conda, Python 3.6, CUDA 8.0)
- tqdm (
conda install -c conda-forge tqdm
)
-
Then, install the correlation package:
./install.sh
-
The datasets used for this projects are followings:
- FlyingChairsOcc dataset (available upon request)
- MPI Sintel Dataset
- KITTI Optical Flow 2015 and KITTI Optical Flow 2012
- FlyingThings3D subset
Training
The scripts
folder contains training scripts of experiments demonstrated in the paper.
To train the model, you can simply run the script file, e.g., ./IRR-PWC_flyingChairsOcc.sh
.
In script files, please configure your own experiment directory (EXPERIMENTS_HOME) and dataset directory in your local system (e.g., SINTEL_HOME or KITTI_HOME).
Pretrained Models
The saved_check_point
contains the pretrained models of i) baseline, ii) baseline + irr, and iii) full models.
Additional pretrained models in the ablations study (Table 1 in the main paper) and their training scripts are available upon request.
Inference
The scripts for testing the pre-trained models are located in scripts/validation
.
Acknowledgement
Portions of the source code (e.g., training pipeline, runtime, argument parser, and logger) are from Jochen Gast