• Laser & Optoelectronics Progress
  • Vol. 60, Issue 16, 1610013 (2023)
Yang Yang, Zhennan Ren*, and Beichen Li
Author Affiliations
  • School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
  • show less
    DOI: 10.3788/LOP222265 Cite this Article Set citation alerts
    Yang Yang, Zhennan Ren, Beichen Li. Infrared and Visible Image Fusion with Convolutional Neural Network and Transformer[J]. Laser & Optoelectronics Progress, 2023, 60(16): 1610013 Copy Citation Text show less
    Overall structure of the proposed model
    Fig. 1. Overall structure of the proposed model
    Concrete structure of the Encoder
    Fig. 2. Concrete structure of the Encoder
    Fusion strategy
    Fig. 3. Fusion strategy
    Fusion results of the “Street” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method
    Fig. 4. Fusion results of the “Street” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method
    Fusion results of the “Kaptein_1123” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method
    Fig. 5. Fusion results of the “Kaptein_1123” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method
    Fusion results of the “Kaptein_1654” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method
    Fig. 6. Fusion results of the “Kaptein_1654” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method
    Fusion results of the “Bunker” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method
    Fig. 7. Fusion results of the “Bunker” image. (a) Infrared image; (b) visible image; (c) RP; (d) Wavelet; (e) ResNet-ZCA; (f) DenseFuse; (g) Dual-Branch; (h) FusionGAN; (i) GANMcC; (j) proposed method
    Six objective metrics of different fusion models on TNO dataset. (a) En; (b) SD; (c) SF; (d) MI; (e) SCD; (f) Q_abf
    Fig. 8. Six objective metrics of different fusion models on TNO dataset. (a) En; (b) SD; (c) SF; (d) MI; (e) SCD; (f) Q_abf
    Subjective results of the ablation experiment
    Fig. 9. Subjective results of the ablation experiment
    ModelEnSDSFMISCDQ_abf
    cnn&mean6.554628.20876.02952.23801.71050.3059
    cnn+trans&mean6.657431.77636.52092.26481.72490.3110
    cnn&spa_fus6.777535.12027.46653.29541.68790.4115
    Proposed method6.963940.72717.95352.69491.76450.3695
    Table 1. Average value of objective results of the ablation experiment
    Yang Yang, Zhennan Ren, Beichen Li. Infrared and Visible Image Fusion with Convolutional Neural Network and Transformer[J]. Laser & Optoelectronics Progress, 2023, 60(16): 1610013
    Download Citation