Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information
-
摘要: 卷积神经网络中的局部卷积运算无法对高光谱影像中的全局语义信息进行充分学习,因此,基于Transformer模型设计了一种新颖的深度网络模型,以进一步提高高光谱影像分类精度。首先,利用主成分分析方法对高光谱影像进行降维处理,并选取像素周围邻域数据作为输入样本,以充分利用影像中的空谱联合信息; 然后,利用卷积层将输入样本转换为序列特征向量; 最后,利用构建的深度Transformer网络进行分类。Transformer模型中的多头注意力机制能够充分利用丰富的判别性信息。试验表明,与现有卷积神经网络模型相比,文章方法能够获得更为优异的分类性能。
-
关键词:
- 高光谱影像分类 /
- Transformer /
- 深度学习 /
- 自注意力机制
Abstract: The local convolution operation in convolutional neural networks cannot fully learn the global semantic information in hyperspectral images. Given this, this study designed a novel deep network model based on Transformer in order to further improve the classification precision of hyperspectral images. Firstly, this study reduced the dimensionality of hyperspectral images using the principal component analysis method and selected the neighborhood data around pixels as input samples to fully utilize the spatial-spectral information in the images. Secondly, the convolutional layer was used to transform the input samples into sequential characteristic vectors. Finally, image classification was conducted using the designed deep Transformer network. The multi-head attention mechanism in the Transformer model can make full use of the rich discriminative information. Experimental results show that the method proposed in this study can achieve better classification performance than the existing convolutional neural network model. -
[1] He L, Li J, Liu C, et al. Recent advances on spectral-spatial hyperspectral image classification:An overview and new guidelines[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(3):1579-1597.
[2] Ghamisi P, Plaza J, Chen Y, et al. Advanced spectral classifiers for hyperspectral images:A review[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(1):8-32.
[3] Tao C, Pan H, Li Y, et al. Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(12):2438-2442.
[4] Li T, Zhang J, Zhang Y. Classification of hyperspectral image based on deep belief networks[C]// Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), 2014.
[5] Zhang X R, Sun Y J, Jiang K, et al. Spatial sequential recurrent neural network for hyperspectral image classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(11):4141-4155.
[6] Xu Q, Xiao Y, Wang D, et al. CSA-MSO3DCNN:Multiscale octave 3D CNN with channel and spatial attention for hyperspectral image classification[J]. Remote Sensing, 2020, 12(1):188.
[7] Gao K, Guo W, Yu X, et al. Deep induction network for small samples classification of hyperspectral images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 3462-3477.
[8] 高奎亮, 张鹏强, 余旭初, 等. 基于Network In Network网络结构的高光谱影像分类方法[J]. 测绘科学技术学报, 2019, 36(5):500-504,510.
[9] Gao K L, Zhang P Q, Yu X C, et al. Classification method of hyperspectral image based on Network In Network structure[J]. Journal of Geomatics Science and Technology, 2019, 36(5):500-504,510.
[10] Li Y, Zhang H, Shen Q. Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network[J]. Remote Sensing, 2017, 9(1):67.
[11] Xu X, Li J, Li S. Multiview intensity-based active learning for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2):669-680.
[12] He X, Chen Y. Transferring CNN ensemble for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(5):876-880.
[13] Mou L, Ghamisi P, Zhu X X. Unsupervised spectral-spatial feature learning via deep residual Conv-Deconv network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(1):391-406.
[14] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]// Thirty-first Conference on Neural Information Processing Systems, 2017.
[15] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16[Z]. Transformers for Image Recognition at Scale, 2020.
[16] Yue J, Zhao W, Mao S, et al. Spectral-spatial classification of hyperspectral images using deep convolutional neural networks[J]. Remote Sensing Letters, 2015, 6(4-6):468-477.
[17] 刘冰, 余旭初, 张鹏强, 等. 联合空-谱信息的高光谱影像深度三维卷积网络分类[J]. 测绘学报, 2019, 48(1):53-63.
[18] Liu B, Yu X C, Zhang P Q, et al. Deep 3D convolutional network combined with spatial-spectral features for hyperspectral image classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):53-63.
-
计量
- 文章访问数: 546
- PDF下载数: 60
下载: