PyTorch implementation of Dual-Camera Smooth Zoom on Mobile Phones
- π₯ Data for ZoomGS and fine-tuning FI models is now available.
- π₯ Codes and pre-trained models are now available.
Overview of the proposed method. (a) Data preparation for data factory. We collect multi-view dual-camera images and calibrate their camera extrinsic and intrinsic parameters. (b) Construction of ZoomGS in data factory. ZoomGS employs a camera transition (CamTrans) module to transform the base (i.e., UW camera) Gaussians to the specific camera Gaussians according to the camera encoding. (c) Data generation from data factory. The virtual (V) camera parameters are constructed by interpolating the dual-camera ones, and are then input into ZoomGS to generate zoom sequences. (d) Fine-tuning a frame interpolation (FI) model with the constructed zoom sequences.
- Python 3.7.13, PyTorch 1.12.1, cuda-11.8
- opencv, numpy, Pillow, timm, tqdm, scikit-image
- We provide detailed dependencies in
environment.yml
Please download data from Baidu Netdisk (Chinese: ηΎεΊ¦η½η).
- Dataset for zoomGS: https://pan.baidu.com/s/1tZ326PQ7SI_LoDAMreXbBQ ζεη : xje3
- Dataset for FI: https://pan.baidu.com/s/1rIaAc2Huprl796qguiB8AQ ζεη : w4zf
-
Pretrained model link: https://pan.baidu.com/s/1_bfNrij8HwtwlON32TiCWg?pwd=x66g ζεη : x66g
-
Please put the above models into './FI/pretrained_dirs'
-
Fretrained model link: https://pan.baidu.com/s/1QeuSrRo4E5dIEMNGiJRLiw ζεη : hya8
-
Please put the above models into './FI/ckpt'
- Run
cd ./ZoomGS
- Run
bash ./zoomgs_train.sh
- Run
cd ./FI
- Training: run
bash ./train.sh
- Testing on synthetic data: run
bash ./test_syn.sh
- Testing on real-world data: run
bash ./test_real.sh
UW Image (x0.6) | W Image (x1.0) | Dual-Camera Smooth Zoom (x0.6->x1.0) |
![]() |
![]() |
|
![]() |
![]() |
|
Special thanks to the following awesome projects!
If you make use of our work, please cite our paper.
@article{DCSZ,
title={Dual-Camera Smooth Zoom on Mobile Phones},
author={Wu, Renlong and Zhang, Zhilu and Yang, Yu and Zuo, Wangmeng},
journal={ECCV},
year={2024}
}