DaFIR: Distortion-Aware Representation Learning for Fisheye Image Rectification
TCSVT 2023
Any questions or discussions are welcomed!
- Dear researchers and engineers, good afternoon. Due to my busy work and study schedule, the DaFIR project has only been fully open-sourced today. Thank you for your patience. Below is a detailed description of this project.
- python 3.8
- cudnn 8.2.1
- pytorch 1.9.0
- cuda 11.2
- numpy 1.22.3
- skimage 0.20.0
- opencv 4.7.0
- Pillow 9.4.0
- timm 0.6.13
- In this project, we synthesize a fisheye image from a group of distortion parameters and a source image without distortion.
- The source images in the size of 256*256 can be download from Baidu Cloud (Extraction code: gjq3 ). Download the file "picture.zip", extract to the fold "picture" and put it into the path /code_dafir/data_prepare/.
- The path list of the above source images can be download from Baidu Cloud (Extraction code: 9p97 ). Download the file "img_path.txt" and put it into the path /code_dafir/data_prepare_ddm/ and /code_dafir/data_prepare_flow/.
-
Synthesize Fisheye Images and DDM (D labels) for pretraining dataset
cd /code_dafir/data_prepare_ddm/ python get_dataset_ddm.py --index 0
The index parameter can be adjust to change different source images for synthesization
-
Synthesize Fisheye Images and Pixel-wise Flow Maps for fine-tuning dataset
cd /code_dafir/data_prepare_flow/ python get_dataset.py --index 0
The index parameter can be adjust to change different source images for synthesization
- Input Fisheye Images and Predict D Labels.
The parameter --master_port can be selected randomly. Use 2 GPUs for distributed trainning.
cd /code_dafir/pre_training/ python -m torch.distributed.launch --nproc_per_node=2 --master_port 1285 main.py
- After Pretraining, the model is saved in /code_dafir/pre_training/save/net/
- Put a Pre-traning release model into the path /code_dafir/fine-tuning/pretrain/
- Input Fisheye Images and Predict Pixel-wise Flow Map.
The parameter --master_port can be selected randomly. Use 2 GPUs for distributed trainning.
cd /code_dafir/fine-tuning/ python -m torch.distributed.launch --nproc_per_node=2 --master_port 1285 main.py
- After fine-tuning, the model is saved in /code_dafir/fine-tuning/save/net/
- We provide a test dataset, which is the same with the one in our paper (https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10251977). This dataset can be download from [Baidu Cloud] (https://pan.baidu.com/s/1xXuCdmgrjGlwQeDIeCMFgQ?pwd=yzv7)(Extraction Code: yzv7). Unrap the fold dataset3 and put it into the path /code_dafir/.
- Testing the model in synthesize fisheye images
cd /code_dafir/fine-tuning/ python test.py
- Testing the model in real fisheye images
cd /code_dafir/fine-tuning/ python test2.py
If you find this code useful for your research, please use the following BibTeX entry.
@article{liao2023dafir,
title={DaFIR: Distortion-aware Representation Learning for Fisheye Image Rectification},
author={Liao, Zhaokang and Zhou, Wengang and Li, Houqiang},
journal={IEEE Transactions on Circuits and Systems for Video Technology},
year={2023},
publisher={IEEE}
}
- Evaluate the model with metric PSNR and SSIM
cd /code_dafir/core/ python compare.py
Any questions please contact me in lzk9508@mail.ustc.edu.cn
The codes are largely based on PCN and MAE. Thanks for their wonderful works.