xiongzhu666/BasicVSR_PlusPlus

Official repository of "BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment"

BasicVSR_PlusPlus (CVPR 2022)

[Paper] [Project Page] [Code]

This is the official repository for BasicVSR++. Please feel free to raise issue related to BasicVSR++! If you are also interested in RealBasicVSR, which is also accepted to CVPR 2022, please don't hesitate to star!

Authors: Kelvin C.K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy, Nanyang Technological University

Acknowedgement: Our work is built upon MMEditing. Please follow and star this repository and MMEditing!

News

• 2 Dec 2021: Colab demo released
• 18 Apr 2022: Code released. Also merged into MMEditing
• 5 Feb 2023: The checkpoints for BasicVSR_2x is released.

TODO

• Add BasicVSR_2x architecture
• Add BasicVSR_2x checkpoints
• Add data processing scripts
• Add checkpoints for deblur and denoise
• Add configs for deblur and denoise
• Add Colab demo

Pre-trained Weights

You can find the pre-trained weights for deblurring and denoising in this link. For super-resolution and compressed video enhancement, please refer to MMEditing.

Installation

1. Install PyTorch
2. pip install openmim
3. mim install mmcv-full
4. git clone https://github.com/ckkelvinchan/BasicVSR_PlusPlus.git
5. cd BasicVSR_PlusPlus
6. pip install -v -e .

Inference a Video

1. Download pre-trained weights
2. python demo/restoration_video_demo.py ${CONFIG} ${CHKPT} ${IN_PATH} ${OUT_PATH}

For example, you can download the VSR checkpoint here to chkpts/basicvsr_plusplus_reds4.pth, then run

python demo/restoration_video_demo.py configs/basicvsr_plusplus_reds4.py chkpts/basicvsr_plusplus_reds4.pth data/demo_000 results/demo_000

You can also replace ${IN_PATH} ${OUT_PATH} by your video path (e.g., xxx/yyy.mp4) to input/output videos.

Training Models

1. Put the dataset in the designated locations specified in the configuration file.
2. sh tools/dist_train.sh ${CONFIG} ${NGPUS}

Data Preprocessing

To be added...

Related Work

Our BasicVSR series:

1. BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond, CVPR 2021
2. Investigating Tradeoffs in Real-World Video Super-Resolution, CVPR 2022

More about deformable alignment:

• Understanding Deformable Alignment in Video Super-Resolution, AAAI 2021

Citations

@inproceedings{chan2022basicvsrpp,
  author = {Chan, Kelvin C.K. and Zhou, Shangchen and Xu, Xiangyu and Loy, Chen Change},
  title = {{BasicVSR++}: Improving video super-resolution with enhanced propagation and alignment},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
  year = {2022}
}

@article{chan2022generalization,
  title={On the Generalization of {BasicVSR++} to Video Deblurring and Denoising},
  author={Chan, Kelvin CK and Zhou, Shangchen and Xu, Xiangyu and Loy, Chen Change},
  journal={arXiv preprint arXiv:2204.05308},
  year={2022}
}