• Laser & Optoelectronics Progress
  • Vol. 60, Issue 16, 1610008 (2023)
Keqi Liu1, Mianmian Dong1,*, Hui Gao1, Zhigang Lü1..., Baoyi Guo2 and Min Pang3|Show fewer author(s)
Author Affiliations
  • 1School of Electronic and Information Engineering, Xi'an Technological University, Xi'an 710021, Shaanxi, China
  • 2Undergraduate College, Xi'an Technological University, Xi'an 710021, Shaanxi, China
  • 3Beijing Institute of Microelectronics Technology, Beijing 100000, China
  • show less
    DOI: 10.3788/LOP222528 Cite this Article Set citation alerts
    Keqi Liu, Mianmian Dong, Hui Gao, Zhigang Lü, Baoyi Guo, Min Pang. Multi-Modal Pedestrian Detection Algorithm Based on Illumination Perception Weight Fusion[J]. Laser & Optoelectronics Progress, 2023, 60(16): 1610008 Copy Citation Text show less
    Structure of the proposed network
    Fig. 1. Structure of the proposed network
    Structure of ECA mechanism module
    Fig. 2. Structure of ECA mechanism module
    Structure of IPWF module
    Fig. 3. Structure of IPWF module
    SKNet structure
    Fig. 4. SKNet structure
    Visible and infrared image pair under shade trees. (a) Visible light image; (b) infrared image
    Fig. 5. Visible and infrared image pair under shade trees. (a) Visible light image; (b) infrared image
    MR-FPPI curves of different fusion strategies
    Fig. 6. MR-FPPI curves of different fusion strategies
    MR-FPPI curves of different algorithms
    Fig. 7. MR-FPPI curves of different algorithms
    Pedestrian target detection results in day time scenes. (a) Original annotation; (b) test result
    Fig. 8. Pedestrian target detection results in day time scenes. (a) Original annotation; (b) test result
    Pedestrian target detection results in night time scenes. (a) Original annotation; (b) test result
    Fig. 9. Pedestrian target detection results in night time scenes. (a) Original annotation; (b) test result
    Pedestrian target detection results on LLVIP dataset. (a) Detection results of daytime condition; (b) detection results of night condition
    Fig. 10. Pedestrian target detection results on LLVIP dataset. (a) Detection results of daytime condition; (b) detection results of night condition
    Pedestrian target detection results on M3FD dataset. (a) Detection results of daytime condition; (b) detection results of night condition
    Fig. 11. Pedestrian target detection results on M3FD dataset. (a) Detection results of daytime condition; (b) detection results of night condition
    Fusion strategyMR /%Model size /106
    All the timeDayNight
    ZJDD15.6617.8511.39305.7
    IPWF12.7414.699.51320.5
    IPWF+CBAM13.3314.8110.17346.3
    IPWF+SE13.1714.509.86346.0
    TIWF+ECA12.1113.659.45317.7
    IPWF+ECA11.1612.408.46320.7
    Table 1. Performance comparison of different fusion policies
    Ablation strategyLclsLregLIMR /%
    Focal lossCross entropy lossSmooth L1CIOUFocal lossCross entropy loss
    Strategy 111.87
    Strategy 212.06
    Strategy 312.17
    Strategy 412.55
    Strategy 511.60
    Strategy 611.16
    Table 2. Experimental results of loss function ablation
    AlgorithmMR /%Speed /(frame·s-1
    All the timeDayNight
    ACF+T+THOG2347.3242.5756.170.27
    Halfway-fusion425.7524.8826.590.43
    Fusion-RPN518.2919.5716.270.80
    IAF-RCNN915.7314.5518.260.21
    IATDNN+IAMSS814.9514.6715.720.25
    GFR2411.5112.6410.630.12
    Proposed algorithm11.1612.408.460.08
    Table 3. Performance comparison of different algorithms
    AlgorithmLLVIP datasetM3FD dataset
    MR /%Speed /(frame·s-1MR /%Speed /(frame·s-1
    ACF+T+THOG2359.400.3251.200.30
    Halfway-fusion434.400.4728.940.43
    Fusion-RPN526.150.8121.100.80
    IAF-RCNN923.700.2018.280.21
    IATDNN+IAMSS824.410.2518.160.26
    GFR2422.100.1417.370.12
    Proposed algorithm20.170.0916.070.08
    Table 4. Performance comparison of different algorithms on LLVIP dataset and M3FD dataset
    Keqi Liu, Mianmian Dong, Hui Gao, Zhigang Lü, Baoyi Guo, Min Pang. Multi-Modal Pedestrian Detection Algorithm Based on Illumination Perception Weight Fusion[J]. Laser & Optoelectronics Progress, 2023, 60(16): 1610008
    Download Citation