/Coarse2Fine-DSIT

Pattern Recognition 2022 // A Coarse-to-Fine Approach for Dynamict-to-static Image Transformation

Primary LanguagePython

Dynamic-to-static Image Transformation

Dataset and code for our Pattern-Recognition paper: 《A Coarse-to-Fine Approach for Dynamic-to-static Image Transformation》

News

Installation

  • Install Torch (tested on 1.2.0)

Dataset

  • Synthetic Dataset: EmptyCities (for train, test and validation)

    this dataset is generated with CARLA 0.8.2 by Berta et al. in IEEE TRO2020 Paper;

    and it is available in this link;

    how to generate dataset by CARLA could be found here;

  • Synthetic Dataset: New (with large dynamic rate range!!)

    this dataset is generated with CARLA 0.8.2 by us for further evaluation; and it is available in this link;

Train

  • First step: train coarse network
python train.py --gpu_ids 0 --batchSize 4 --netG unet_256 --netD basic 
--mode Coarse --name CoarseNet_unet8_load400
  • Second step: end-to-end train coarse-to-fine network
python train.py --gpu_ids 0 --batchSize 4 --netG Coarse2fineNet --netD SA 
--mode Coarse2fine --name Coarse2fineNet_unet8_1206
  • Visualization on TensorBoard for training is supported.
tensorboard --logdir model_logs --port 6006

Test

  • test coarse network
python test.py --phase test --gpu_ids 0 --eval --no_flip --netG unet_256 
--mode Coarse --name CoarseNet_unet8_load400 --which_epoch 21
  • test coarse2fine network
python test.py --phase test --gpu_ids 0 --eval --no_flip --netG Coarse2fineNet 
--mode Coarse2fine --name Coarse2fineNet_unet8_1206 --which_epoch 42

Transfer to Real Data

  • load pretrained model (i.e. which epoch) on CARLA synthetic dataset, then continue train from next one epoch by an appropriate learning rate on Cityscapes Dataset.
python train.py --gpu_ids 0 --batchSize 1 --lr 0.0001 --netG Coarse2fineNet --netD SA 
--mode Transfer --name transferModel_0614 --continue_train --which_epoch 42 --epoch_count 43
  • test
python test.py --phase val --gpu_ids 0 --eval --no_flip --netG Coarse2fineNet 
--mode  Transfer --name transferModel_0614 --which_epoch 42

Resource

Related Resource in my BaiduCloud(Extraction-code:5250)

Citation

@article{WANG2022108373,
title = {A coarse-to-fine approach for dynamic-to-static image translation},
journal = {Pattern Recognition},
volume = {123},
pages = {108373},
year = {2022},
issn = {0031-3203},
doi = {https://doi.org/10.1016/j.patcog.2021.108373},
url = {https://www.sciencedirect.com/science/article/pii/S0031320321005537},
author = {Teng Wang and Lin Wu and Changyin Sun},

.

Acknowledge