Specral-spatial classification of hyperspectral imagery with hybrid architecture of 3D-CNN and Transformer

Haizhao JING; Lijie TAO; Haokui ZHANG

doi:10.37188/OPE.20243223.3504

Journals >Optics and Precision Engineering >Volume 32 >Issue 23 >Page 3504 > Article

Optics and Precision Engineering
Vol. 32, Issue 23, 3504 (2024)

Specral-spatial classification of hyperspectral imagery with hybrid architecture of 3D-CNN and Transformer

Haizhao JING, Lijie TAO, and Haokui ZHANG^*

Author Affiliations

Northwestern Polytechnical University， Xi’an710129， China

show less

DOI: 10.37188/OPE.20243223.3504 Cite this Article

Haizhao JING, Lijie TAO, Haokui ZHANG. Specral-spatial classification of hyperspectral imagery with hybrid architecture of 3D-CNN and Transformer[J]. Optics and Precision Engineering, 2024, 32(23): 3504 Copy Citation Text

show less

Fig. 1. Workflow of 3D-ConvFormer framework

Download full size | View in the Article

Fig. 2. Operation of 3D-ConvFormer block

Download full size | View in the Article

Fig. 3. Visualization results of different methods on Indian Pine dataset

Download full size | View in the Article

Fig. 4. Visualization results of different methods on Pavia University dataset

Download full size | View in the Article

Fig. 5. Visualization results of different methods on WHU-Hi-LongKou dataset

Download full size | View in the Article

Fig. 6. Scatter plot of overall accuracy predicted by five methods on three datasets

Download full size | View in the Article

Class	Train num	Test num
Alfalfa	46	10
Corn-notill	1 428	150
Corn-mintill	830	150
Corn	237	10
Grass-pasture	483	150
Grass-trees	730	150
Grass-pasture-mowed	28	10
Hay-windrowed	478	150
Oats	20	10
Soybean-notill	972	150
Soybean-mintill	2 455	150
Soybean-clean	593	150
Wheat	205	10
Woods	1 265	150
Buildings-Grass-Trees-Drives	386	150
Stone-Steel-Towers	93	10
Total	10 249	1 560

Table 1. Land cover class in Indian Pines dataset

View in the Article

Class	Train num	Test num
Asphalt	6 631	150
Meadows	18 649	150
Gravel	2 099	150
Trees	3 064	150
Painted metal sheets	1 345	150
Bare Soil	5 029	150
Bitumen	1 330	150
Self-Blocking Bricks	3 682	150
Shadows	947	150
Total	42 776	1 350

Table 2. Land cover class in Pavia University dataset

View in the Article

Class	Train num	Test num
Corn	34 511	150
Cotton	8 374	150
Sesame	3 031	150
Broad-leaf soybean	63 212	150
Narrow-leaf soybean	4 151	150
Rice	11 854	150
Water	67 056	150
Roads and houses	7 124	150
Mixed weed	5 229	150
Total	204 542	1 350

Table 3. Land cover class in WHU-Hi-LongKou dataset

View in the Article

Class	3D-CNN	LWNet	SpectralFormer	SSFTT	3D-ConvFormer
C1	97.22	100.00	97.83	100.00	91.67
C2	97.81	98.28	91.39	95.07	95.93
C3	98.82	99.85	96.02	100.00	99.56
C4	93.39	82.82	47.68	92.51	91.63
C5	100.00	100.00	98.34	99.70	100.00
C6	100.00	97.59	98.90	99.48	100.00
C7	100.00	100.00	100.00	100.00	100.00
C8	100.00	100.00	100.00	100.00	100.00
C9	100.00	100.00	100.00	100.00	100.00
C10	97.93	99.39	94.34	99.15	99.03
C11	98.57	99.05	84.97	99.22	99.13
C12	99.77	97.52	98.31	95.26	99.77
C13	85.64	82.56	71.22	85.64	89.74
C14	100.00	99.91	98.81	100.00	99.28
C15	100.00	100.00	98.70	100.00	100.00
C16	86.75	92.77	87.10	78.31	95.18
OA（%）	98.37	98.22	91.98	96.52	98.41
AA（%）	97.24	96.86	91.48	97.97	97.56
KAPPA（%）	98.11	97.94	90.89	97.65	98.16

Table 4. Comparison of classification results of different methods on Indian Pines dataset

View in the Article

Class	3D-CNN	LWNet	SpectralFormer	SSFTT	3D-ConvFormer
C1	96.62	93.18	88.80	96.76	99.15
C2	94.37	97.71	84.10	99.34	99.70
C3	91.74	97.69	79.66	98.92	99.69
C4	96.12	93.45	94.88	81.67	97.25
C5	100.00	99.83	99.78	98.24	99.92
C6	99.80	100.00	91.41	99.92	100.00
C7	99.92	99.92	95.34	99.49	100.00
C8	96.72	99.04	89.35	99.52	98.58
C9	97.62	97.24	99.79	91.09	99.37
OA（%）	95.94	97.20	87.88	96.11	99.39
AA（%）	96.99	97.56	91.45	97.57	99.30
KAPPA（%）	94.62	96.26	84.34	96.74	99.18

Table 5. Comparison of classification results of different methods on Pavia University dataset

View in the Article

Class	3D-CNN	LWNet	SpectralFormer	SSFTT	3D-ConvFormer
C1	98.00	97.05	98.91	99.56	99.66
C2	98.20	98.54	60.45	99.29	99.77
C3	99.79	99.93	98.81	100.00	99.90
C4	91.47	93.18	84.90	98.02	96.29
C5	99.58	99.93	86.27	99.48	99.63
C6	98.79	98.97	98.76	98.10	98.89
C7	98.64	98.38	98.86	96.81	99.73
C8	92.63	92.47	93.95	95.23	98.48
C9	98.41	97.99	98.41	96.63	98.39
OA（%）	96.12	96.42	92.53	98.12	98.53
AA（%）	97.28	97.38	91.03	97.68	98.97
KAPPA（%）	94.94	95.32	90.37	97.20	98.07

Table 6. Comparison of classification results of different methods on WHU-Hi-LongKou dataset

Download Citation

Save the article for my favorites

Paper Information

微信扫一扫：分享

微信扫一扫：分享