This repo contains the code, results and evaluations of the letter ''Context-based Multi-scale Unified Network for Missing Data Reconstruction in Remote Sensing Images'' in GRSL.
Missing data reconstruction is a classical yet challenging problem in remote sensing images. Most current methods based on traditional Convolutional Neural Network require supplementary data and can only handle one specific task. To address these limitations, we propose a novel Generative Adversarial Network-based missing data reconstruction method in this letter, which is capable of various reconstruction tasks given only single source data as input. Two auxiliary patch-based discriminators are deployed to impose additional constraints on the local and global region, respectively. In order to better fit the nature of remote sensing images, we introduce special convolutions and attention mechanism in a two-stage generator, thereby benefiting the tradeoff between accuracy and efficiency. Combining with perceptual and multi-scale adversarial losses, the proposed model can produce coherent structure with better details. Qualitative and quantitative experiments demonstrate the uncompromising performance of the proposed model against multi-source methods in generating visually plausible reconstruction results. Moreover, further exploration shows a promising way for the proposed model to utilize spatio-spectral-temporal information.
Mask data: DATA
We also provide a simple tool make_list.py. Unzip and run it in the source folder.
Run tensorboard --logdir model_logs --port 6006 to view training progress.
Here are several evaluation indexes training on RSSCN7 dataset with first 15K epochs.
One can also visualize the diagrams with their own data and settings.
It is a model for inpainting task on remote sensing images.
The idea is inspired with Global & Local, GatedConv, SA-GAN and Perceptual Loss.
Conda environment with Pytorch
Tensorboardhere may require Tensorflow
Try train.py and test.py.
GMacs = 1.52 + 0.38 + 39.64 = 41.54
Param Number = 4.94 + 4.94 + 6.05 = 15.93M
One can also evaluate any model by running ./flops_count.py.
- We deactivate the local discriminator for SLC-off problem in
./models/sa_gan.py. - One can train the model on other data and settings in
./config/inpaint_sagan.yml. - We'll upload a pretrained model asap.
If you find this work useful for your research, please cite us:
@article{shao2020context,
title={Context-based multiscale unified network for missing data reconstruction in remote sensing images},
author={Shao, Mingwen and Wang, Chao and Wu, Tianjun and Meng, Deyu and Luo, Jiancheng},
journal={IEEE Geoscience and Remote Sensing Letters},
volume={19},
pages={1--5},
year={2020}
}
Please contact me if there is any question. (Chao Wang oliversavealien@gmail.com)