anhuipl2010/HVI-CIDNet

"You Only Need One Color Space: An Efficient Network for Low-light Image Enhancement"

 

You Only Need One Color Space: An Efficient Network for Low-light Image Enhancement

Yixu Feng^∗ , Cheng Zhang^∗ , Pei Wang , Peng Wu , Qingsen Yan , Yanning Zhang

 

💡News

• 2024.04.18 Full version Code, models, visual comparison have been released. We promise that all the weights only trained on train sets, and each weights is reproducible. Hope it will help your future work. If you have trained a better result, please contact us. We look forward to subsequent work based on the HVI color "space"! 💎

• 2024.04.14 Update test fine tuning result and weights on LOLv1 dataset. 🧾

• 2024.03.04 Update five unpaired datasets (DICM, LIME, MEF, NPE, VV) visual results. ✨

• 2024.03.03 Update pre-trained weights and output results on our HVI-CIDNet using Baidu Pan. 🧾

• 2024.02.08 Update HVI-CIDNet in Arxiv. The new code, models and results will be uploaded. 🎈

⚙ Proposed HVI-CIDNet

HVI-CIDNet pipeline:

CID dual-branch UNet structure:

🖼 Visual Comparison

LOL-v1, LOL-v2-real, and LOL-v2-synthetic:

DICM, LIME, MEF, NPE, and VV:

LOL-Blur:

🧾Weights and Results

All the weights that we trained on different datasets is available at [Baidu Pan] (code: yixu). Results on DICM, LIME, MEF, NPE, and VV datasets can be downloaded from [Baidu Pan] (code: yixu). Bolded fonts represent impressive metrics.

• The metrics of HVI-CIDNet on paired datasets are shown in the following table:

Folder (test datasets)	PSNR	SSIM	LPIPS	GT Mean	Results	Weights Path
(LOLv1) v1 w perc loss/ wo gt mean	23.8091	0.8574	0.0856		Baidu Pan	LOLv1/w_perc.pth
(LOLv1) v1 w perc loss/ w gt mean	27.7146	0.8760	0.0791	√	ditto	LOLv1/w_perc.pth
(LOLv1) v1 wo perc loss/ wo gt mean	23.5000	0.8703	0.1053		Baidu Pan	LOLv1/wo_perc.pth
(LOLv1) v1 wo perc loss/ w gt mean	28.1405	0.8887	0.0988	√	ditto	LOLv1/wo_perc.pth
(LOLv2_real) v2 wo perc loss/ wo gt mean	23.4269	0.8622	0.1691		Baidu Pan	(lost)
(LOLv2_real) v2 wo perc loss/ w gt mean	27.7619	0.8812	0.1649	√	ditto	(lost)
(LOLv2_real) v2 best gt mean	28.1387	0.8920	0.1008	√	Baidu Pan	LOLv2_real/w_prec.pth
(LOLv2_real) v2 best Normal	24.1106	0.8675	0.1162		Baidu Pan	(lost)
(LOLv2_real) v2 best PSNR	23.9040	0.8656	0.1219		Baidu Pan	LOLv2_real/best_PSNR.pth
(LOLv2_real) v2 best SSIM	23.8975	0.8705	0.1185		Baidu Pan	LOLv2_real/best_SSIM.pth
(LOLv2_real) v2 best SSIM/ w gt mean	28.3926	0.8873	0.1136	√	None	LOLv2_real/best_SSIM.pth
(LOLv2_syn) syn wo perc loss/ wo gt mean	25.7048	0.9419	0.0471		Baidu Pan	LOLv2_syn/wo_perc.pth
(LOLv2_syn) syn wo perc loss/ w gt mean	29.5663	0.9497	0.0437	√	ditto	LOLv2_syn/wo_perc.pth
(LOLv2_syn) syn w perc loss/ wo gt mean	25.1294	0.9388	0.0450		Baidu Pan	LOLv2_syn/w_perc.pth
(LOLv2_syn) syn w perc loss/ w gt mean	29.3666	0.9500	0.0403	√	ditto	LOLv2_syn/w_perc.pth
Sony_Total_Dark	22.9039	0.6763	0.4109		Baidu Pan	SID.pth
LOL-Blur	26.5719	0.8903	0.1203		Baidu Pan	LOL-Blur.pth
SICE-Mix	13.4235	0.6360	0.3624	√	Baidu Pan	SICE.pth
SICE-Grad	13.4453	0.6477	0.3181	√	Baidu Pan	SICE.pth

• Performance on five unpaired datasets are shown in the following table:

metrics	DICM	LIME	MEF	NPE	VV
NIQE	3.79	4.13	3.56	3.74	3.21
BRISQUE	21.47	16.25	13.77	18.92	30.63

• While we don't recommend that you perform finetuning on the test set, in order to demonstrate the effectiveness of our model, we also provide here the results of test finetuning training on the LOLv1 dataset. Using the fine tuning technique on the test set does make the PSNR metrics higher, but other metrics are not found to be significantly changed on CIDNet, which may result in a lower generalization of the model, so we do not recommend you do this.

Folder (test datasets)	PSNR	SSIM	LPIPS	GT Mean	Results	Weights Path
(LOLv1) v1 test finetuning	25.4036	0.8652	0.0897		Baidu Pan	LOLv1/test_finetuning.pth
(LOLv1) v1 test finetuning	27.5969	0.8696	0.0869	√	ditto	ditto

1. Get Started 🌑

Dependencies and Installation

• Python 3.7.0
• Pytorch 1.13.1

(1) Create Conda Environment

conda create --name CIDNet python=3.7.0
conda activate CIDNet

(2) Clone Repo

git clone git@github.com:Fediory/HVI-CIDNet.git

(3) Install Dependencies

cd HVI-CIDNet
pip install -r requirements.txt

Data Preparation

You can refer to the following links to download the datasets. Note that we only use low_blur and high_sharp_scaled subsets of LOL-Blur dataset.

• LOLv1
• LOLv2
• LOL-Blur(code: yixu)
• DICM,LIME,MEF,NPE,VV(code: yixu)
• SICE(code: yixu)
• Sony-Total-Dark(code: yixu)

Then, put them in the following folder:

datasets (click to expand)

├── datasets
    ├── DICM
    ├── LIME
    ├── LOLv1
        ├── our485
            ├──low
            ├──high
	    ├── eval15
            ├──low
            ├──high
    ├── LOLv2
        ├── Real_captured
            ├── Train
                ├── Low
                ├── Normal
	        ├── Test
                ├── Low
                ├── Normal
        ├── Synthetic
            ├── Train
                ├── Low
                ├── Normal
	        ├── Test
                ├── Low
                ├── Normal
    ├── LOL_blur
        ├── eval
            ├── high_sharp_scaled
            ├── low_blur
        ├── test
            ├── high_sharp_scaled
                ├── 0012
                ├── 0017
                ...
            ├── low_blur
                ├── 0012
                ├── 0017
                ...
        ├── train
            ├── high_sharp_scaled
                ├── 0000
                ├── 0001
                ...
            ├── low_blur
                ├── 0000
                ├── 0001
                ...
    ├── MEF
    ├── NPE
    ├── SICE
        ├── Dataset
            ├── eval
                ├── target
                ├── test
            ├── label
            ├── train
                ├── 1
                ├── 2
                ...
        ├── SICE_Grad
        ├── SICE_Mix
        ├── SICE_Reshape
    ├── Sony_total_dark
        ├── eval
            ├── long
            ├── short
        ├── test
            ├── long
                ├── 10003
                ├── 10006
                ...
            ├── short
                ├── 10003
                ├── 10006
                ...
        ├── train
            ├── long
                ├── 00001
                ├── 00002
                ...
            ├── short
                ├── 00001
                ├── 00002
                ...
    ├── VV

2. Testing 🌒

Download our weights from [Baidu Pan] (code: yixu) and put them in folder weights:

├── weights
    ├── LOLv1
        ├── w_perc.pth
        ├── wo_perc.pth
        ├── test_finetuning.pth
    ├── LOLv2_real
        ├── best_PSNR.pth
        ├── best_SSIM.pth
        ├── w_perc.pth
    ├── LOLv2_syn
        ├── w_perc.pth
        ├── wo_perc.pth
    ├── LOL-Blur.pth
    ├── SICE.pth
    ├── SID.pth

• You can test our method as followed, all the results will saved in ./output folder:

# LOLv1
python eval.py --lol --perc # weights that trained with perceptual loss
python eval.py --lol # weights that trained without perceptual loss

# LOLv2-real
python eval.py --lol_v2_real --best_GT_mean # you can choose best_GT_mean or best_PSNR or best_SSIM

# LOLv2-syn
python eval.py --lol_v2_syn --perc # weights that trained with perceptual loss
python eval.py --lol_v2_syn # weights that trained without perceptual loss

# SICE
python eval.py --SICE_grad # output SICE_grad
python eval.py --SICE_mix # output SICE_mix

# Sony-Total-Dark
python eval_SID_blur --SID

# LOL-Blur
python eval_SID_blur --Blur

# five unpaired datasets DICM, LIME, MEF, NPE, VV. 
# We note that: you can choose one weights in ./weights folder, and set the alpha float number (defualt=1.0) as illumination scale of the datasets.
# You can change "--DICM" to the other unpaired datasets "LIME, MEF, NPE, VV".
python eval.py --unpaired --DICM --unpaired_weights --alpha
# e.g.
python eval.py --unpaired --DICM --unpaired_weights ./weights/LOLv2_syn/w_perc.pth --alpha 0.9

• Also, you can test all the metrics mentioned in our paper as follows:

# LOLv1
python measure.py --lol

# LOLv2-real
python measure.py --lol_v2_real

# LOLv2-syn
python measure.py --lol_v2_syn

# Sony-Total-Dark
python measure_SID_blur.py --SID

# LOL-Blur
python measure_SID_blur.py --Blur

# SICE-Grad
python measure.py --SICE_grad

# SICE-Mix
python measure.py --SICE_mix


# five unpaired datasets DICM, LIME, MEF, NPE, VV. 
# You can change "--DICM" to the other unpaired datasets "LIME, MEF, NPE, VV".
python measure_niqe_bris.py --DICM


# Note: Following LLFlow, KinD, and Retinxformer, we have also adjusted the brightness of the output image produced by the network, based on the average value of GroundTruth (GT). This only works in paired datasets. If you want to measure it, please add "--use_GT_mean".
# 
# e.g.
python measure.py --lol --use_GT_mean
  

• Evaluating the Parameters, FLOPs, and running time of HVI-CIDNet:

python net_test.py

3. Training 🌓

• We put all the configurations that need to be adjusted in the ./data/options.py folder and explained them in the file. We apologize that some of the training parameters we are no longer able to provide and share with you, but we guarantee that all the weights are trainable by parameter tuning. You can train our HVI-CIDNet by:

python train.py

• All weights are saved to the ./weights/train folder and are saved in steps of the checkpoint set in the options.py as epoch_*.pth where * represent the epoch number.
• Also, for every weight saved, metrics are measured for the validation set and printed to the command line. Finally, the results of all weights' test metrics on the validation set and options in ./data/options.py will be saved to ./results/training/metrics.md.
• In each epoch, we save an output (test) and GT image to the ./results/training folder to facilitate the visualization of the training results and progress of each epoch, as well as to detect the generation of gradient explosion in advance.
• After each checkpoint, we save all the validation set outputs for this time in the ./results folder to the corresponding folder. Note that we use a replacement strategy for different checkpoints for each dataset. That is, we do not save the plots of all checkpoints, but only the weights of each checkpoint.

4. Contacts 🌔

If you have any questions, please contact us or submit an issue to the repository!

Yixu Feng (yixu-nwpu@mail.nwpu.edu.cn) Cheng Zhang (zhangcheng233@mail.nwpu.edu.cn)

5. Citation

If you find our work useful for your research, please cite our paper

@misc{feng2024need,
      title={You Only Need One Color Space: An Efficient Network for Low-light Image Enhancement}, 
      author={Yixu Feng and Cheng Zhang and Pei Wang and Peng Wu and Qingsen Yan and Yanning Zhang},
      year={2024},
      eprint={2402.05809},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}