Hyperspectral super-resolution combining multi-receptive field features with spectral-spatial attention
-
摘要: 针对高光谱图像超分辨率过程中,图像细节信息容易丢失的问题,提出多感受野特征与空谱注意力结合的高光谱图像超分辨率算法,该算法充分利用高光谱图像中的高频信息与低频信息,减少图像细节信息丢失,提升了高光谱图像超分辨率效果。首先,在特征提取阶段采用不同大小卷积核的卷积,获取到多尺度感受野特征,更好地提取低分辨率图像中的高频信息与低频信息,有助于保留原始图像的特征信息; 然后,把获取到图像特征,经过“空间-光谱“结合的注意力机制增强,利用光谱维信息辅助空间维特征重建; 最后,把每组的特征融合,通过像素级反卷积层缓解棋盘格效应,输出清晰的高分辨率图像。实验结果表明: 提出的多感受野特征与空谱注意力结合的超分辨率算法在Chikusei和Pavia center scene这2个公开数据集上峰值信噪比分别达到了39.8697和31.9422,结构相似度分别达到了0.9376和0.8786,与最新超分辨率算法比较有明显的性能优势。该文提出的算法,结合了多感受野特征提取模块和空谱结合注意力模块的优势,超分辨率后的图像细节特征有了明显的改善。Abstract: To address the problem that image details are liable to be lost in the process of hyperspectral super-resolution, this study proposed a hyperspectral super-resolution algorithm that combines multi-receptive field features and spectral-spatial attention. By fully using the high- and low-frequency information in hyperspectral images, this algorithm reduces the loss of image details and improves the hyperspectral super-resolution effects. First, in the feature extraction stage, convolution with different sizes of convolutional kernels is used to obtain multi-scale receptive field features. This assists in extracting more high- and low-frequency information from low-resolution images, thus retaining the features of original images. Then, the acquired image features are enhanced by the spatial-spectral attention mechanism, and the reconstruction of spatial-dimension features is conducted using spectral-dimension information. Finally, the features of various groups are fused, and the checkerboard pattern is relieved by applying the pixel deconvolution layer. As a result, clear and high-resolution images can be produced. The proposed super-resolution algorithm that combines multi-receptive field features with spectral-spatial attention was applied to two public datasets Chikusei and Pavia Center Scene, achieving peak signal-to-noise ratios of 39.869 7 and 31.942 2, respectively and structural similarity of 0.937 6 and 0.878 6, respectively. Therefore, the super-resolution algorithm enjoys obvious performance advantages compared to the latest super-resolution algorithms. Overall, the algorithm proposed in this study integrates the advantages of the multi-receptive field feature extraction module and the spatial-spectral attention module and can significantly improve image details.
-
-
[1] Muhammad U, Arif M, Ajmal M. Hyperspectral face recognition using 3D-DCT and partial least squares[C]// Burghardt: BMVA Press.UK:University of Bristol,2013:57.1- 57.10.[2] Lowe A, Harrison N, French A P. Hyperspectral image analysis techniques for the detection and classification of the early onset of plant disease and stress[J]. Plant Methods, 2017, 13(1):80-92. [3] Lin J, Clancy N T, Qi J, et al. Dual-modality endoscopic probe for tissue surface shape reconstruction and hyperspectral imaging enabled by deep neural networks[J]. Medical Image Analysis, 2018, 48:162-176. [4] Akhtar N, Shafait F, Mian A S. Sparse spatio-spectral representation for hyperspectral image super-resolution[C]// European Conference on Computer Vision, 2014:63-78.[5] Dian R, Li S, Guo A, et al. Deep hyperspectral image sharpening[J]. IEEE Transactions on Neural Network Learning Systems, 2018, 29(11):5345-5355. [6] Huang H, Yu J, Sun W. Super-resolution mapping via multi-dictionary based sparse representation[C]// IEEE International Conference on Acoustic,Speech and Signal Processing, 2014:3523-3527.[7] 练秋生, 张钧芹, 陈书贞. 基于两级字典与分频带字典的图像超分辨率算法[J]. 自动化学报, 2013, 39(8):1310-1320.[7] Lian Q S, Zhang J Q, Chen S Z. Image superresolution algorithm based on two-level dictionary and subband dictionary[J]. Acta Automatica Sinica, 2013, 39(8):1310-1320. [8] Yang J, Wright J, Huang T S, et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11):2861-2873. [9] Li X, Wei H W, Zhang H Q. Super-resolution reconstruction of single remote sensing image combined with deep learning[J]. Journal of Image and Graphics, 2018, 23(2):209-218.[10] Qiao J, Song H, Zhang K, et al. Image super-resolution using conditional generative adversarial network[J]. IET Image Processing, 2019, 13(14):2673-2679. [11] Pan J T, Kevin M G, Elisa S, et al. Shallow and deep convolutional networks for saliency prediction[C]// IEEE Conference on Computer Vision and Pattern Recognition, 2016:598-606.[12] Li K, Dai D X, Ender K, et al. Hyperspectral image super-resolution with spectral mixup and heterogeneous datasets[EB/OL].(2021-1-19)[2021-3-8].https://arxiv.org/pdf/2101.07589.htm. [13] Aggarwal H K, Majumdar A. Hyperspectral image denoising using spatio-spectral total variation[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(3):442-446.[14] Loncan L, De Almeida L B, Briottet X, et al. Hyperspectral pansharpening[J]. IEEE Geoscience and Remote Sensing Magazine, 2015, 3(3):27-46. [15] Wald L. Data fusion:Definitions and architectures:Fusion of images of different spatial resolutions[M]. Presses des MINES,Computer Science, 2002.[16] Peng J Y, Shi C Y, Laugeman E, et al. Implementation of the structural SIMilarity (SSIM) index as a quantitative evaluation tool for dose distribution error detection[J]. Medical Physics, 2020, 47(4):1907-1919. [17] Mei S, Yuan X, Ji J, et al. Hyperspectral image spatial super-resolution via 3D full convolutional neural network[J]. Remote Sensing, 2017, 9(11):1139. [18] 李成轶, 田淑芳. 基于字典学习的遥感影像超分辨率融合方法[J]. 自然资源遥感, 2017, 29(1):50-56.doi: 10.6046/gtzyyg.2017.01.08. [18] Li C Y, Tian S F. Super-resolution fusion method for remote sensing image based on dictionary learning[J]. Remote Sensing for Land and Resource, 2017, 29(1):50-56.doi: 10.6046/gtzyyg.2017.01.08. [19] Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016:1646-1654.[20] Yuan Y, Zheng X, Lu X. Hyperspectral image superresolution by transfer learning[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(5):1963-1974. [21] Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017:136-144.[22] Zhang Y, Li K P, Li K, et al. Image super-resolution using very deep residual channel attention networks[C]// Proceedings of the European Conference on Computer Vision (ECCV), 2018:286-301.[23] Dai T, Cai J, Zhang Y, et al. Second-order attention network for single image super-resolution[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2019:11065-11074.[24] Mei S, Yuan X, Ji J, et al. Hyperspectral image spatial super-resolution via 3D full convolutional neural network[J]. Remote Sensing, 2017, 9(11):1139. [25] Li Y, Zhang L, Dingl C, et al. Single hyperspectral image super-resolution with grouped deep recursive residual network[C]// Proceedings of the IEEE International Conference on Multimedia Big Data, 2018:1-4.[26] Sidorov O, Hardeberg J Y. Deep hyperspectral prior:Single-image denoising,inpainting,super-resolution[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop, 2019:3844-3851.
-
计量
- 文章访问数: 973
- PDF下载数: 49
- 施引文献: 0
下载: