Multiscale-Super-Spectral

Accurate Spectral Super-resolution from Single RGB Image Using Multi-scale CNN

Cite our work

@inproceedings{yan2018accurate,
title={Accurate Spectral Super-Resolution from Single RGB Image Using Multi-scale CNN},
author={Yan, Yiqi and Zhang, Lei and Li, Jun and Wei, Wei and Zhang, Yanning},
booktitle={Chinese Conference on Pattern Recognition and Computer Vision (PRCV)},
pages={206--217},
year={2018},
organization={Springer}
}

Project webpabe

Pre-trained models

Download pre-trained model from Google Drive
Place them at TestLog.

Contents of this repository

dataset.py: data pre-processing
model.py & train.py: our model
model_ref.py & train_ref.py: the comparison model
test.py: test the models and report the evaluation results
utilities.py: some auxiliary functions

How to run

1. Dependences

PyTorch (<0.4)
OpenCV for Python3
HDF5 for Python
tensorboardX
Matplotlib

2. Download data

NTIRE 2018 challenge on spectral reconstruction from RGB images (Track 1)

Place the data at data and arrange the directories as follows:

data/Train_Spectral/
--BGU_HS_00001.mat
... ...
--BGU_HS_00256.mat

data/Train_RGB/
--BGU_HS_00001_clean.png
... ...
--BGU_HS_00256_clean.png

data/Test_Spectral/
--BGU_HS_00257.mat
--BGU_HS_00259.mat
--BGU_HS_00261.mat
--BGU_HS_00263.mat
--BGU_HS_00265.mat

data/Test_RGB/
--BGU_HS_00257_clean.png
--BGU_HS_00259_clean.png
--BGU_HS_00261_clean.png
--BGU_HS_00263_clean.png
--BGU_HS_00265_clean.png

3. Train our model

If you are running the code for the first time, remember data pre-processing.

python3 train.py --preprocess True

Otherwise, run training directly.

python3 train.py

You may also adjust hyper-parameters such as batch size, initial learning rate, dropout rate, etc.

4. Train the comparison model

Reference paper: Learned Spectral Super-resolution
The data pre-processing is exactly the same as above, so you can run training directly.

python3 train_ref.py

5. Test our model

Dropout rate: 0.2

python3 test.py --model Model \
  --dropout 2

Dropout rate: 0

python3 test.py --model Model \
  --dropout 0

6. Test the comparison model

Dropout rate: 0.5

python3 test.py --model Ref \
  --dropout 5