Semantic Segmentation of Dual-Source Remote Sensing Images Based on Gated Attention and Multiscale Residual Fusion

Wen Guo; Hong Yang; Chang Liu

doi:10.3788/LOP240534

Journals >Laser & Optoelectronics Progress >Volume 61 >Issue 18 >Page 1837015 > Article

Laser & Optoelectronics Progress
Vol. 61, Issue 18, 1837015 (2024)

Semantic Segmentation of Dual-Source Remote Sensing Images Based on Gated Attention and Multiscale Residual Fusion

Wen Guo¹, Hong Yang¹, and Chang Liu^2,*

Author Affiliations

¹School of science, Beijing Information Science and Technology University, Beijing 100029, China

²Institute of Applied Mathematics, Beijing Information Science and Technology University, Beijing 100101, China

show less

DOI: 10.3788/LOP240534 Cite this Article Set citation alerts

Wen Guo, Hong Yang, Chang Liu. Semantic Segmentation of Dual-Source Remote Sensing Images Based on Gated Attention and Multiscale Residual Fusion[J]. Laser & Optoelectronics Progress, 2024, 61(18): 1837015 Copy Citation Text

show less

Fig. 1. Overall structure diagram of STAM-SegNet

Fig. 2. AGCA module

Fig. 3. AGSA module

Fig. 4. MFRF module

Fig. 5. Overall technical flowchart of the experiment

Download full size

Fig. 6. Corresponding prediction effect diagram of comparative experimental results of different models on the Vaihingen dataset

Download full size

Fig. 7. Corresponding prediction effect diagram of comparative experimental results of different models on the Potsdam dataset

Download full size

Item	ISPRS Vaihingen	ISPRS Potsdam
Input size	512 $\times$ 512	512 $\times$ 512
Bands used	IRRG，DSM	IRRG，DSM
Batch size	4	4
Train number	16	17
Val number	4	5
Test number	17	14

Table 1. Dataset preprocessing and training configuration

Baseline（Swin Transformer）	AGCA	AGSA	MFRF	Mean F1 /%	OA /%
√	×	×	×	88.59	89.36
√	√	×	×	89.08	89.94
√	√	√	×	89.45	90.47
√	√	√	√	89.66	90.73

Table 2. Comparison of evaluation results of ablation experiment on the Vaihingen test set

Method	Input	Mean F1 /%	OA /%
Baseline	IRRG	88.02	88.94
Add	IRRG+DSM	88.73	89.52
Concat	IRRG+DSM	88.87	89.77
AGCA	IRRG+DSM	89.66	90.73

Table 3. Comparative experimental results of dual-source data feature fusion on the Vaihingen dataset

Method	Backbone	F1-score /%					Mean F1 /%	OA /%	GFLOPs /GB	Params /10⁶
Method	Backbone	Impervios_surface	Building	Low_vegetable	Tree	Car	Mean F1 /%	OA /%	GFLOPs /GB	Params /10⁶
DeepLabV3+	Resnet101	91.52	94.08	81.67	88.64	82.47	87.68	89.24	193.94	49.58
UperNet	Resnet101	91.48	93.89	81.21	88.34	86.14	88.21	89.07	243.97	70.60
DANet	Resnet101	92.15	94.80	83.19	89.17	86.54	89.17	90.01	276.73	66.45
TransUNet	Resnet101+VIT	92.08	94.91	83.02	88.98	86.57	89.11	89.95	803.4	90.7
Swin-UNet	Swin Transformer	92.49	95.64	83.12	89.31	86.37	89.38	90.44	349.72	82.89
STAM-SegNet	Swin Transformer	92.56	95.96	83.82	89.34	86.61	89.66	90.73	233.21	60.47

Table 4. Comparison results of multiple evaluation metrics for each model on the Vaihingen testset

Method	Backbone	F1-score /%					Mean F1 /%	OA /%	GFLOPS /GB	Params /10⁶
Method	Backbone	Impervios_surface	Building	Low_vegetable	Tree	Car	Mean F1 /%	OA /%	GFLOPS /GB	Params /10⁶
DeepLabV3+	Resnet101	92.08	95.65	86.16	87.13	95.65	91.33	89.67	193.94	49.58
UperNet	Resnet101	92.52	95.98	86.38	87.45	95.85	91.64	90.01	243.97	70.60
DANet	Resnet101	92.60	96.16	86.55	87.26	95.90	91.69	90.10	276.73	66.45
TransUNet	Resnet101+VIT	92.34	96.10	86.23	87.29	96.03	91.60	90.07	803.4	90.7
Swin-UNet	Swin Transformer	92.91	96.91	87.15	88.37	95.99	92.27	90.72	349.72	82.89
STAM-SegNet	Swin Transformer	93.58	97.13	87.51	89.19	96.32	92.75	91.23	233.21	60.47

Table 5. Comparison results of multiple evaluation metrics for each model on the Potsdam testset

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information

微信扫一扫：分享

微信扫一扫：分享