ningbende/imagefusion_densefuse

Infrared and visible image fusion using CNN layers and dense block architecture. (IEEE TIP 2019)

Densefuse: A Fusion Approach to Infrared and Visible Images - Tensorflow

Published in: IEEE Transactions on Image Processing

H. Li, X. J. Wu, “DenseFuse: A Fusion Approach to Infrared and Visible Images,” IEEE Trans. Image Process., vol. 28, no. 5, pp. 2614–2623, May. 2019.

• IEEEXplore
• arXiv

Note

In 'main.py' file, you will find how to run these codes.

The evaluate methods which used in our paper are shown in 'analysis_MatLab'. And these methods are implemented by MatLab.

Abstract

In this paper, we present a novel deep learning architecture for infrared and visible images fusion problem.

In contrast to conventional convolutional networks, our encoding network is combined by convolutional neural network layer and dense block which the output of each layer is connected to every other layer. We attempt to use this architecture to get more useful features from source images in encoder process. Then appropriate fusion strategy is utilized to fuse these features. Finally, the fused image is reconstructed by decoder.

Compare with existing fusion methods, the proposed fusion method achieves state-of-the-art performance in objective and subjective assessment.

The framework of fusion method

Fusion strategy - addition

Fusion strategy - l1-norm

Training

We train our network using MS-COCO 2014(T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014. 3-5.) as input images which contains 80000 images and all resize to 256×256 and RGB images are transformed to gray ones. Learning rate is 1×10^(-4). The batch size and epochs are 2 and 4, respectively. Our method is implemented with GTX 1080Ti and 64GB RAM.

Experimental results

Infrared and visible images('street')

Infrared and visible images(RGB)

Database:
Hwang S, Park J, Kim N, et al. Multispectral pedestrian detection: Benchmark dataset and baseline[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 1037-1045.

Multi-focus images(RGB)

If you have any question about this code, feel free to reach me(hui_li_jnu@163.com, lihui@stu.jiangnan.edu.cn)

Citation

For paper:

H. Li, X. J. Wu, “DenseFuse: A Fusion Approach to Infrared and Visible Images,” IEEE Trans. Image Process., vol. 28, no. 5, pp. 2614–2623, May. 2019.

For code:

@misc{li2018IVimagefusion_densefuse,
    author = {Hui Li},
    title = {CODE: DenseFuse_A Fusion Approach to Infrared and Visible Images},
    year = {2018},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{https://github.com/hli1221/imagefusion_densefuse}}
  }