• Laser & Optoelectronics Progress
  • Vol. 60, Issue 2, 0210003 (2023)
Haitao Yin* and Wei Zhou
Author Affiliations
  • College of Automation and College of Artificial Intelligence, Nanjing University of Posts and Telecommunications, Nanjing 210023, Jiangsu, China
  • show less
    DOI: 10.3788/LOP212488 Cite this Article Set citation alerts
    Haitao Yin, Wei Zhou. Multi-Scale Dilated Convolutional Neural Network Based Multi-Focus Image Fusion Algorithm[J]. Laser & Optoelectronics Progress, 2023, 60(2): 0210003 Copy Citation Text show less
    Residual block
    Fig. 1. Residual block
    SE attention module
    Fig. 2. SE attention module
    Architecture of MDF-Net
    Fig. 3. Architecture of MDF-Net
    Multi-scale dilated block
    Fig. 4. Multi-scale dilated block
    Dilated convolutional with different dilation rates.(a)Dilation rate is 1;(b)dilation rate is 2;
    Fig. 5. Dilated convolutional with different dilation rates.(a)Dilation rate is 1;(b)dilation rate is 2;
    Examples of simulated multi-focus images
    Fig. 6. Examples of simulated multi-focus images
    Fused results of “children” image. (a)(b) Source images; (c) NSCT; (d) SR; (e) IMF; (f) MWGF; (g) CNN; (h) DeepFuse; (i) DenseFuse-ADD; (j) DenseFuse-L1; (k) IFCNN-MAX; (l) MDF-Net
    Fig. 7. Fused results of “children” image. (a)(b) Source images; (c) NSCT; (d) SR; (e) IMF; (f) MWGF; (g) CNN; (h) DeepFuse; (i) DenseFuse-ADD; (j) DenseFuse-L1; (k) IFCNN-MAX; (l) MDF-Net
    Fusion results of “monkey” image. (a)(b) Source images; (c) NSCT; (d) SR; (e) IMF; (f) MWGF; (g) CNN; (h) DeepFuse; (i) DenseFuse-ADD; (j) DenseFuse-L1; (k) IFCNN-MAX; (l) MDF-Net
    Fig. 8. Fusion results of “monkey” image. (a)(b) Source images; (c) NSCT; (d) SR; (e) IMF; (f) MWGF; (g) CNN; (h) DeepFuse; (i) DenseFuse-ADD; (j) DenseFuse-L1; (k) IFCNN-MAX; (l) MDF-Net
    Fusion results of “gymnasium” image. (a)(b) Source images; (c) NSCT; (d) SR; (e) IMF; (f) MWGF; (g) CNN; (h) DeepFuse; (i) DenseFuse-ADD; (j) DenseFuse-L1; (k) IFCNN-MAX; (l) MDF-Net
    Fig. 9. Fusion results of “gymnasium” image. (a)(b) Source images; (c) NSCT; (d) SR; (e) IMF; (f) MWGF; (g) CNN; (h) DeepFuse; (i) DenseFuse-ADD; (j) DenseFuse-L1; (k) IFCNN-MAX; (l) MDF-Net
    Fusion results of “statue” image. (a)(b) Source images; (c) NSCT; (d) SR; (e) IMF; (f) MWGF; (g) CNN; (h) DeepFuse; (i) DenseFuse-ADD; (j) DenseFuse-L1; (k) IFCNN-MAX; (l) MDF-Net
    Fig. 10. Fusion results of “statue” image. (a)(b) Source images; (c) NSCT; (d) SR; (e) IMF; (f) MWGF; (g) CNN; (h) DeepFuse; (i) DenseFuse-ADD; (j) DenseFuse-L1; (k) IFCNN-MAX; (l) MDF-Net
    AlgorithmAGSFVIFQAB/F
    NSCT6.253018.70750.91000.6685
    SR6.246318.86900.91560.7096
    IMF6.280618.78510.87950.6973
    MWGF6.049217.44560.84220.6906
    CNN6.179518.52870.92680.7165
    DeepFuse4.172010.96470.70620.5375
    DenseFuse-ADD4.507112.09230.80490.6110
    DenseFuse-L14.454611.84920.78440.6084
    IFCNN-MAX6.420718.96270.94210.6876
    MDF-Net6.454119.34780.94240.7098
    Table 1. Indexes values of various fusion algorithms on "gymnasium" image
    AlgorithmAGSFVIFQAB/F
    NSCT6.850618.80120.91870.7168
    SR6.832818.99400.93090.7437
    IMF6.968819.22980.93190.7406
    MWGF6.844619.01560.93080.7437
    CNN6.881419.00050.93730.7555
    DeepFuse4.075110.41870.66130.4860
    DenseFuse-ADD4.467611.51750.77270.5806
    DenseFuse-L14.361611.03730.75100.5587
    IFCNN-MAX6.874218.97180.93270.7105
    MDF-Net6.995719.29760.94370.7488
    Table 2. Average indexes values of various fusion algorithms on Lytro dataset
    Variants of MDF-NetAGSFVIFQAB/F
    Without-Dilated6.808818.80460.91320.7485
    Without-SE6.945919.17780.93720.7472
    Without-Cat6.824818.80410.92130.7468
    MDF-Net6.995719.29760.94370.7488
    Table 3. Indexes results of different variants of MDF-Net
    #MDBAGSFVIFQAB/F
    16.783318.71250.92170.7465
    26.921919.07690.93340.7473
    36.995719.29760.94370.7488
    46.764318.62640.91250.7443
    Table 4. Ablation experiment on number of MDB modules
    #BranchAGSFVIFQAB/F
    26.749018.57000.90350.7484
    36.824318.79550.91690.7446
    46.995719.29760.94370.7488
    56.909418.94950.93010.7403
    Table 5. Ablation experiment on number of dilated convolution branches in MDB
    #ChannelAGSFVIFQAB/F
    64-64-646.861618.89610.92690.7457
    128-128-1286.889118.91240.92350.7455
    256-256-2566.831918.80760.92940.7401
    64-128-2566.995719.29760.94370.7488
    Table 6. Ablation experiment on number of feature channels in MDB
    Haitao Yin, Wei Zhou. Multi-Scale Dilated Convolutional Neural Network Based Multi-Focus Image Fusion Algorithm[J]. Laser & Optoelectronics Progress, 2023, 60(2): 0210003
    Download Citation