• Laser & Optoelectronics Progress
  • Vol. 60, Issue 2, 0228011 (2023)
Lu Xiong, Zhenwen Deng, Wei Tian*, and Zhiang Wang
Author Affiliations
  • School of Automotive Studies, Tongji University, Shanghai 201804, China
  • show less
    DOI: 10.3788/LOP220712 Cite this Article Set citation alerts
    Lu Xiong, Zhenwen Deng, Wei Tian, Zhiang Wang. Three-Dimensional Pedestrian Detection by Fusing Image Semantics and Point Cloud Spatial Visibility Features[J]. Laser & Optoelectronics Progress, 2023, 60(2): 0228011 Copy Citation Text show less
    Schematic of ray traversing grid
    Fig. 1. Schematic of ray traversing grid
    Logic diagram of 2D Raycasting algorithm
    Fig. 2. Logic diagram of 2D Raycasting algorithm
    Overall framework of fusion network
    Fig. 3. Overall framework of fusion network
    Semantic segmentation and point feature enhancement
    Fig. 4. Semantic segmentation and point feature enhancement
    Geometric feature and semantic feature encoding
    Fig. 5. Geometric feature and semantic feature encoding
    Spatial visibility feature encoding
    Fig. 6. Spatial visibility feature encoding
    Feature fusion and detection heads
    Fig. 7. Feature fusion and detection heads
    Visibility feature visualization. (a) BEV of point cloud; (b) single layer feature
    Fig. 8. Visibility feature visualization. (a) BEV of point cloud; (b) single layer feature
    Comparison results of pedestrian detection (example 1). (a) (c) Benchmark results; (b) (d) results obtained by proposed method
    Fig. 9. Comparison results of pedestrian detection (example 1). (a) (c) Benchmark results; (b) (d) results obtained by proposed method
    Comparison results of pedestrian detection (example 2). (a) (c) Benchmark results; (b) (d) results obtained by proposed method
    Fig. 10. Comparison results of pedestrian detection (example 2). (a) (c) Benchmark results; (b) (d) results obtained by proposed method

    Visibility code

    [U,O,F]

    3D pedestrian detection AP /%mAP /%
    EasyModerateHard
    [0,1,-1]68.7662.4957.7463.00
    [0.5,0.7,0.4]70.8464.5759.7765.06
    Table 1. Performance comparison of different visibility description methods on KITTI dataset
    Number of channels3D pedestrian detection AP /%mAP /%
    0.5 m1.0 m2.0 m4.0 m
    162.2464.3666.3668.7865.44
    3269.6871.8773.7375.9772.81
    Table 2. Performance comparison of different density along height direction on nuScenes dataset
    Nunber of frames3D pedestrian detection AP /%mAP /%
    0.5 m1.0 m2.0 m4.0 m
    137.4638.2239.2940.4038.84
    1068.3670.6372.3074.5971.47
    Table 3. Performance comparison of different number of frames on nuScenes dataset
    Method3D pedestrian detection AP /%mAP /%
    EasyModerateHard
    PP(Baseline)70.1663.4057.4963.68
    PP+Vis.71.8664.4958.7265.02
    PP+Img.72.0865.5660.3465.99
    PP+Vis.+Img.71.8465.6560.8166.10
    Table 4. Performance comparison of different optimized methods on KITTI validation set
    Method3D pedestrian detection AP /%Speed /Hz
    EasyModerateHard
    VoxelNet739.4833.6931.514.4
    AVOD336.1027.8625.7610
    SECOND1851.0742.5637.2920
    F-PointNet450.5342.1538.085.9
    PointPainting650.3240.9737.872.5
    PointRCNN1947.9839.3736.0110
    Proposed method51.0741.3637.8330
    Table 5. Performance comparison of different methods on KITTI test set
    Lu Xiong, Zhenwen Deng, Wei Tian, Zhiang Wang. Three-Dimensional Pedestrian Detection by Fusing Image Semantics and Point Cloud Spatial Visibility Features[J]. Laser & Optoelectronics Progress, 2023, 60(2): 0228011
    Download Citation