Optimal LED Spectral Multiplexing for NIR2RGB Translation

Lei Lui, Yuze Chen, Junchi Yan, Yinqiang Zheng

This paper has been accpected by CVPR2022. In this short tutorial, we will guide you through setting up the system environment for running the code, which used for NIR-to-RGB translation.

Requirments

Ubuntu 16.04
CUDA 9.1
pytorch 1.7.1

Datasets

We have released our hyperspectral images dataset IDH, the wavelength range from 420nm to 1000nm with 10nm intervals.
If you only want to go through our model, we suggest to download the processed Dataset and unzip it into datasets/. More details see datasets/readme.txt.

Target Loss Minimization (TLM)

Training phase

Take ICVL for example:

python train.py --dataroot path/to/the/datasets/icvl/train --name experiment_name

On training image outputs and model are stored in checkpoints/experiment_name, if you have multi GPUs, using --gpu_ids 0 to specify the gpu you want to use.

Testing phase

First, make sure that the data in datasets/icvl/test folder is avaliable.
Pretrained model:

Dataset	Camera	Model Link
ICVL	FLIR GS3-U3-15S5C	model

After the training step, or download the pretrained model and put them in checkpoints/experiment_name folder. Run the following command to translate NIR images to RGB images:
```
python test.py --dataroot path/to/the/datasets/icvl/test --name experiment_name
```

The results are stored in results/experiment_name folder.

RGB Variance Maximization (RVM)

Run the following command to see the results of RVM with 3 cameras: python util/rvm.py