Medical Image Fusion Based on Semisupervised Learning and Generative Adversarial Network

Haitao Yin; Yongying Yue

doi:10.3788/LOP202259.2215005

Journals >Laser & Optoelectronics Progress >Volume 59 >Issue 22 >Page 2215005 > Article

Laser & Optoelectronics Progress
Vol. 59, Issue 22, 2215005 (2022)

Medical Image Fusion Based on Semisupervised Learning and Generative Adversarial Network

Haitao Yin^* and Yongying Yue

Author Affiliations

College of Automation ＆ College of Artificial Intelligence, Nanjing University of Posts and Telecommunications, Nanjing 210023, Jiangsu , China

show less

DOI: 10.3788/LOP202259.2215005 Cite this Article Set citation alerts

Haitao Yin, Yongying Yue. Medical Image Fusion Based on Semisupervised Learning and Generative Adversarial Network[J]. Laser & Optoelectronics Progress, 2022, 59(22): 2215005 Copy Citation Text

show less

Fig. 1. Schematic diagram of proposed semisupervised learning

Download full size

Fig. 2. Architecture of generator network

Download full size

Fig. 3. SE channel attention module

Download full size

Fig. 4. Architecture of discriminator network

Download full size

Fig. 5. Fused images of unsupervised learning and semisupervised learning. (a) MRI-T1 image; (b) MRI-T2 image; (c) (d) CT images; (e) (f) fused images of unsupervised training; (g) (h) fused images of semisupervised training

Download full size

Fig. 6. Fused results of MRI-T1 and MRI-T2 images. (a) MRI-T1; (b) MRI-T2; (c) U2Fusion; (d) DDcGAN; (e) Deepfuse; (f) DIDFuse; (g) FusionGAN; (h) PF-GAN; (i) SSL-FWGAN

Download full size

Fig. 7. Fused results of MRI-T1 and CT images. (a) MRI-T1; (b) CT; (c) U2Fusion; (d) DDcGAN; (e) Deepfuse; (f) DIDFuse; (g) FusionGAN; (h) PF-GAN; (i) SSL-FWGAN

Download full size

Fig. 8. Fused results of MRI-T2 and CT images. (a) MRI-T1; (b) CT; (c) U2Fusion; (d) DDcGAN; (e) Deepfuse; (f) DIDFuse; (g) FusionGAN; (h) PF-GAN; (i) SSL-FWGAN

Download full size

Fig. 9. Implementation of SSL-FWGAN for fusing MRI and PET images

Download full size

Fig. 10. Fused results of MRI-T1 and PET images. (a) MRI-T1; (b) PET; (c) U2Fusion; (d) DDcGAN; (e) Deepfuse; (f) DIDFuse; (g) FusionGAN; (h) PF-GAN; (i) SSL-FWGAN

Download full size

Fig. 11. Fused results of MRI-T2 and PET images. (a) MRI-T1; (b) PET; (c) U2Fusion; (d) DDcGAN; (e) Deepfuse; (f) DIDFuse; (g) FusionGAN; (h) PF-GAN; (i) SSL-FWGAN

Download full size

Training strategy	Fused result of MRI-T1 and CT images				Fused result of MRI-T2 and CT images
Training strategy	MI	SF	$Q_{y a n g}$	AG	MI	SF	$Q_{y a n g}$	AG
Unsupervised training	0.65	30.92	0.39	8.41	0.61	28.81	0.33	8.31
Semisupervised training	0.72	31.48	0.80	8.94	0.78	30.61	0.74	8.61

Table 1. Index results of unsupervised training and semisupervised training

Method	MRI-T1 and MRI-T2 images				MRI-T1 and CT images				MRI-T2 and CT images
Method	MI	SF	$Q_{y a n g}$	AG	MI	SF	$Q_{y a n g}$	AG	MI	SF	$Q_{y a n g}$	AG
Deepfuse	0.88	20.58	0.38	5.28	0.93	19.29	0.34	2.84	0.97	16.94	0.35	4.06
FusionGAN	0.60	14.62	0.29	4.44	0.61	11.96	0.23	4.46	0.67	10.27	0.21	2.54
PF-GAN	0.68	33.58	0.35	7.76	0.67	26.53	0.37	6.06	0.69	29.19	0.32	5.75
DDcGAN	0.61	33.51	0.35	7.92	0.60	30.94	0.37	3.65	0.54	30.84	0.26	8.74
DIDFuse	0.75	28.14	0.45	6.71	0.74	25.65	0.36	4.34	0.71	24.72	0.36	6.35
U2Fusion	0.71	24.83	0.34	6.75	0.68	22.52	0.37	6.04	0.69	20.73	0.37	5.79
SSL-FWGAN	0.76	33.74	0.76	9.32	0.74	32.41	0.79	8.69	0.74	32.34	0.75	9.48

Table 2. Index results of different methods for fusing gray images

Method	MRI-T1 and PET images				MRI-T2 and PET images
Method	MI	SF	$Q_{y a n g}$	AG	MI	SF	$Q_{y a n g}$	AG
Deepfuse	0.91	23.71	0.49	7.81	0.91	13.09	0.47	4.37
FusionGAN	0.43	14.09	0.29	4.94	0.49	10.68	0.29	3.40
PF-GAN	0.54	34.26	0.51	10.66	0.60	18.59	0.42	6.28
DDcGAN	0.47	43.59	0.54	13.50	0.52	26.49	0.34	8.33
DIDFuse	0.47	23.44	0.26	7.52	0.67	16.00	0.40	5.05
U2Fusion	0.55	26.85	0.34	9.13	0.64	15.24	0.27	4.40
SSL-FWGAN	0.56	47.28	0.57	15.56	0.49	27.09	0.44	9.21

Table 3. Index results of different methods for fusing color images

Haitao Yin, Yongying Yue. Medical Image Fusion Based on Semisupervised Learning and Generative Adversarial Network[J]. Laser & Optoelectronics Progress, 2022, 59(22): 2215005

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information

微信扫一扫：分享

微信扫一扫：分享