A list of papers and resources for spectral reconstruction from images. This page will continue to be updated and will upload our latest research results.
This page mainly describes the overview of spectral reconstruction from RGB images. This work conducts a thorough investigation of more than 25 state-of-the-art spectral reconstruction methods with respect to available database.
The overall taxonomy of the spectral reconstruction methods.
| Dataset | Amount | Resolution | Spectral channels | Spectrum/(nm) | Featured scenes |
|---|---|---|---|---|---|
| CAVE | 32 | 31 | 400-700 | skin, hair, food and drink | |
| ICVL | 203 | 31 | 400-700 | urban, rural, indoor and plant | |
| BGU-HS | 286 | 31 | 400-700 | urban, rural, indoor and plant | |
| ARAD-HS | 510 | 31 | 400-700 | statue, vehicle and paint |
| Method | Category | Priors |
|---|---|---|
| Sparse Coding | Dictionary Learning | sparsity |
| SR A+ | Dictionary Learning | sparsity, local euclidean linearity |
| Multiple Non-negative Sparse Dictionaries | Dictionary Learning | spatial structure similarity, spectral correlation |
| Local Linear Embedding Sparse Dictionary | Dictionary Learning | color and texture, local linearity |
| Spatially Constrained Dictionary Learning | Dictionary Learning | spatial context |
| SR Manifold Mapping | Manifold Learning | low-dimensional manifold |
| SR Gaussian Process | Gaussian Process | spectral physics, spatial structure similarity |
This kind of network is stacked convolutional layers, and the design has only one path and does not include multiple branches. (a) HSCNN (b) SR2D/3DNet (c) Residual HSRCNN
The U-Net model is composed of an encoder and a decoder. (a) SRUNet (b) SRMSCNN (c) SRMXRUNet (d) SRBFWU-Net with supervised learning and unsupervised learning.
The generative adversarial network (GAN) model is composed of a generator and a discriminator. (a) SRCGAN takes Conditional GAN as the main framework. (b) SAGAN includes SAP-UNet withoutboundary supervision and SAP-WNet with boundary supervision.
The core idea of dense network is to densely con-nect all front and back layers to achieve higher flexibil-ity and richer feature representation, which can reducethe vanishing of gradients and ensure the stability ofthe network. (a) SRTiramisuNet (b) HSCNN+
Compared with the linear CNN, the residual network can avoid the vanishing of the gradient by further deepening the network. (a) SREfficientNet (b) SREfficientNet+
The attention-based model allows this flexibility and takesinto account that not all features are important for SR.(a) SRAWAN (b) SRHRNet (c) SRRPAN
In contrast to single-stream CNN, the goal of multi-branch network is to obtain a diverse set of features on multiple context scales.This design can alsoachieve multi-path signal flow, leading to better infor-mation exchange during training. (a) SRLWRDNet (b) SRPFMNet
- Arad, Boaz, and Ohad Ben-Shahar. "Sparse Recovery of Hyperspectral Signal from Natural RGB Images." In ECCV, 2016.[Paper]
- Aeschbacher, Jonas, Jiqing Wu, and Radu Timofte. "In Defense of Shallow Learned Spectral Reconstruction from RGB Images." In ICCVW, 2017.[Paper][code]
- Fu, Ying, et al. "Spectral Reflectance Recovery from A Single RGB Image." IEEE Transactions on Computational Imaging, 2018.[paper]
- Li, Yuqi, Chong Wang, and Jieyu Zhao. "Locally Linear Embedded Sparse Coding for Spectral Reconstruction from RGB Images." IEEE Signal Processing Letters, 2017.[paper]
- Geng, Yunhao, et al. "Spatial Constrained Hyperspectral Reconstruction from RGB Inputs Using Dictionary Representation." IGARSS ,2019.[paper]
- Jia, Yan, et al. "From RGB to Spectrum for Natural Scenes via Manifold-based Mapping." In ICCV, 2017.[paper]
- Akhtar, Naveed, and Ajmal Mian. "Hyperspectral Recovery from RGB Images Using Gaussian Processes." IEEE transactions on pattern analysis and machine intelligence, 2018.[paper]
- Xiong, Zhiwei, et al. "HSCNN: CNN-based Hyperspectral Image Recovery from Spectrally Undersampled Projections." In ICCVW, 2017.[paper]
- Koundinya, Sriharsha, et al. "2D-3D CNN based Architectures for Spectral Reconstruction from RGB Images." In ICCVW, 2018.[paper]
- Han, Xian-Hua, Boxin Shi, and Yinqiang Zheng. "Residual HSRCNN: Residual Hyperspectral Reconstruction CNN from an RGB Image." In ICPR, 2018.[paper]
- Stiebel, Tarek, et al. "Reconstructing Spectral Images from RGB Images Using a Convolutional Neural Network."In ICCVW, 2018.[paper]
- Yan, Yiqi, et al. "Accurate Spectral Super-resolution from Single RGB Image Using Multi-scale CNN." In PRCV, 2018.[paper][code]
- Banerjee, Atmadeep, and Akash Palrecha. "Mxr-u-nets for Real Time Hyperspectral Reconstruction." arXiv, 2020.[paper][code]
- Fubara, Biebele Joslyn, Mohamed Sedky, and David Dyke. "RGB to Spectral Reconstruction via Learned Basis Functions and Weights." In CVPRW, 2020.[paper]
- Alvarez-Gila, Aitor, Joost Van De Weijer, and Estibaliz Garrote. "Adversarial Networks for Spatial Context-aware Spectral Image Reconstruction from RGB." In CVPRW, 2018.[paper]
- Liu, Pengfei, and Huaici Zhao. "Adversarial Networks for Scale Feature-Attention Spectral Image Reconstruction from a Single RGB." Sensors, 2020.[paper]
- Galliani, Silvano, et al. "Learned Spectral Super-resolution." arXiv ,2017.[paper]
- Shi, Zhan, et al. "HSCNN+: Advanced CNN-based Hyperspectral Recovery from RGB Images." IN CVPRW, 2018.[paper][code]
- Can, Yigit Baran, and Radu Timofte. "An Efficient CNN for Spectral Reconstruction from RGB Images." arXiv, 2018.[paper]
- Kaya, Berk, Yigit Baran Can, and Radu Timofte. "Towards Spectral Estimation from a Single RGB Image in the Wild." In ICCVW, 2019.[paper]
- Li, Jiaojiao, et al. "Adaptive Weighted Attention Network with Camera Spectral Sensitivity Prior for Spectral Reconstruction from RGB Images." In CVPRW, 2020.[paper][code]
- Zhao, Yuzhi, et al. "Hierarchical Regression Network for Spectral Reconstruction from RGB Images." In CVPRW, 2020.[paper][code]
- Peng, Hao, Xiaomei Chen, and Jie Zhao. "Residual Pixel Attention Network for Spectral Reconstruction from RGB Images." In CVPRW, 2020.[paper]
- Nathan, D. Sabari, et al. "Light Weight Residual Dense Attention Net for Spectral Reconstruction from RGB Images." arXiv, 2020.[paper]
- Zhang, Lei, et al. "Pixel-aware Deep Function-mixture Network for Spectral Super-resolution." In AAAI, 2020.[paper]