The bathymetry super resolution model is based on a pre-trained SR-ResNet model to generate remote-sensing images with higher resolution than the original ones with limited training dataset. First we pretrain a SR-ResNet model by Large-scale CelebFaces Attributes (CelebA) Dataset which contains 202,599 JPEG images. Then we use transfer learning to adapt our bathymetry dataset composed of 2000 Tiff images selected from ETOPO-1 dataset.) and GEBCO dataset, with fine tuning on the last layer of the model. Finally the model is used to generate high-resolution images for any Tiff inputs. Both the pretrained model and bathymetry model are developed using python API and libraries of PyTorch. The figure below illustrated the overall architecture of our bathymetry super resolution model.
-
The dataset is public and can be downloaded at the official website. Unzip the folder as 'img_align_celeba' at the data folder.
-
The images for training set of our work are manually cropped from ETOPO-1 and GEBCO grid datasets. Please download and unzip in 'data/train': https://drive.google.com/file/d/1ZwW2sGMgVgya-vDHF5xViM2pa0ziapUF/view?usp=share_link.
-
The images for test set of our work are manually cropped from ETOPO-1 and GEBCO grid datasets. Please download and unzip in 'data/test': https://drive.google.com/file/d/16IoRbHVasGl07ZRrQIq11l6eop7N_a3d/view?usp=share_link
$ git clone https://github.com/big-data-lab-umbc/bathymetry_super_resolution
$ cd bathymetry_super_resolution/
$ sudo pip3 install -r requirements.txt
-
We use transfer learning to gain some knowledge of super resolution from RGB images because we have only limited training samples. To implement the transfer learning based super resolution for Bathymetry data, we use a pre-training and fine-tuning approach. In particular, transfer learning with the help of a large-scale CelebFaces Attributes dataset is first performed for the external learning of backbone structure and mapping.
Please refer to README in pre-training folder for training steps.
-
In bathymetry training process, our customized loss functions are combinations of some or all of three parts: content loss, water loss and coastal loss. We have 5 models of different combinations:
- Model 1: content loss
- Model 2: content loss + water loss
- Model 3: water loss
- Model 4: water loss + coastal loss
- Model 5: coastal loss
Please refer to README in Bathymetry training folder for training steps.
The saved trained model by bathymetry training can be used to generate high-resolution images. Load the saved .pth file into generator to test the saved model.
The test images are evaluated by MSE, PSNR and SSIM on the whole area, ocean area and coastal area, respectively. Please use codes in evaluation folder.
The image below illustrates the visual comparison among different experiments and ground truth at the same location. From (a) to (j) are original low-resolution image, bi-cubic interpolation (baseline 1), pre-trained model (baseline 2), directly-trained model (baseline 2), model 1 (content loss), model 2 (content + water loss), model 3 (water loss). model 4 (water + coastal loss), model 5 (coastal loss) and original high-resolution image (ground truth). Among all the experiments, model 5 (coastal loss) is the best model we have. The area in white color has positive pixel values and represents land area, the black area represents deep ocean and the area in grey and around land area represents coastal area.
Our work is designed to conduct super resolution of bathymetry data based on theories and network architecture proposed by paper of SR-GAN: Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. We developed based on a PyTorch version implementation published at Github repository. The detailed information is as below.
Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, Wenzhe Shi