Mengxiang Lin1, Xiuping Huang1, Zhiwei Lin1,2,3,4,*, Sidi Hong5, and Jinfu Liu1,2
Author Affiliations
1College of Computer and Information Science, Fujian Agriculture and Forestry University, Fuzhou 350002, Fujian, China2Key Laboratory for Ecology and Resource Statistics of Fujian Province, Cross-Strait Nature Reserve Research Center, Fujian Agriculture and Forestry University, Fuzhou 350002, Fujian, China3College of Forestry, Fujian Agriculture and Forestry University, Fuzhou 350002, Fujian, China4Forestry Post-Doctoral Station, Fujian Agriculture and Forestry University, Fuzhou 350002, Fujian, China5New Engineering Industry College, Putian University, Putian 351100, Fujian, Chinashow less
Fig. 1. Schematic diagram of precipitation intensity recognition model fusing encoded and decoded features
Fig. 2. Pseudocode of precipitation intensity recognition model
Fig. 3. Rainfall images of each rainfall intensity. (a) Scattered light rain; (b) light rain; (c) moderate rain; (d) heavy rain; (e) rainstorm; (f) heavy rainstorm
Fig. 4. Training and testing loss curves. (a) Training loss; (b) testing loss
Fig. 5. Feature maps of each branch in multi-receptive field convolution. (a) Branch 1; (b) branch 2; (c) branch 3; (d) branch 4
Fig. 6. Heat map of the proposed model under each rainfall intensity. (a) Scattered light rain; (b) light rain; (c) moderate rain; (d) heavy rain; (e) rainstorm; (f) heavy rainstorm
| Layer | Parameters of layer | | Output size |
|---|
| Input | | | 224×224×3 | | Conv | 7×7 Conv，64，stride 2 | | 112×112×64 | | Pool | 3×3，maxpool，stride 2 | | 56×56×64 | | MRF-Conv | Branch 1 | 1×1 Conv，96，stride 1 | Concatenate | 56×56×384 | | Branch 2 | 1×1 Conv，64，stride 1 3×3 Conv，96，stride 1 | | Branch 3 | 1×1 Conv，64，stride 1 3×3 Conv，96，stride 1 3×3 Conv，96，stride 1 | | Branch 4 | 3×3，avgpool，stride 1 1×1 Conv，96，stride 1 | | Pool | 2×2，maxpool，stride 2 | | 28×28×384 | | MRF-Conv. | As above | | 28×28×384 | | Pool | 2×2，maxpool，stride 2 | | 14×14×384 | | Upsample | 2×2 Deconv，64，stride 2 | | 28×28×64 | | MRF-Conv | As above | | 28×28×384 | | Fusion block | Concatenate 3×3 Conv，96，stride 1 | | 28×28×96 | | Upsample | 2×2 Deconv，64，stride 2 | | 56×56×64 | | MRF-Conv | As above | | 56×56×384 | | Fusion block | Concatenate 3×3 Conv，96，stride 1 | | 56×56×96 | | CNN | Resnet or DenseNet | | 7×7×1024 or 7×7×2048 | | FC | Global average pool， FC | | 1×1×6 |
|
Table 1. Parameters of each layer ofproposed model
| Model | Class |
|---|
| Scattered light rain /% | Light rain /% | Moderate rain /% | Heavy rain /% | Rainstorm /% | Heavy rainstorm /% | Overall accuracy /% | Parameter /106 |
|---|
| ResNet-50 | 92.9 | 94.1 | 85.8 | 56.8 | 72.0 | 0.0 | 89.4 | 23.52 | | ResNet-101 | 96.2 | 93.4 | 87.3 | 67.2 | 72.0 | 33.3 | 90.5 | 42.51 | | ResNet-152 | 92.4 | 91.9 | 82.6 | 49.6 | 76.0 | 0.0 | 87.0 | 58.16 | | Proposed-ResNet-50 | 97.1 | 94.5 | 89.0 | 66.4 | 76.0 | 0.0 | 91.7 | 27.28 | | Proposed-ResNet-101 | 95.0 | 91.7 | 86.6 | 62.4 | 84.0 | 66.7 | 89.2 | 46.27 | | Proposed-ResNet-152 | 93.3 | 91.1 | 83.6 | 60.8 | 84.0 | 33.3 | 87.6 | 61.91 | | DenseNet-63 | 93.1 | 89.7 | 86.2 | 50.4 | 84.0 | 0.0 | 87.2 | 2.31 | | DenseNet-121 | 93.8 | 91.0 | 84.8 | 56.8 | 76.0 | 0.0 | 87.8 | 6.96 | | DenseNet-169 | 92.4 | 90.7 | 82.8 | 61.6 | 84.0 | 0.0 | 87.1 | 12.49 | | Proposed-DenseNet-63 | 93.8 | 91.7 | 90.9 | 50.4 | 80.0 | 0.0 | 89.8 | 6.07 | | Proposed-DenseNet-121 | 94.0 | 91.6 | 88.7 | 46.4 | 72.0 | 33.3 | 88.8 | 10.74 | | Proposed-DenseNet-169 | 89.8 | 92.1 | 87.2 | 50.4 | 60.0 | 0.0 | 88.0 | 16.27 |
|
Table 2. Recognition results of different rainfall intensities based on framework of ResNet and DenseNet
| ResNet-50 | Encoder-decoder | Fusion | Accuracy /% | DenseNet-63 | Encoder-decoder | Fusion | Accuracy /% |
|---|
| Concat- once | Concat-twice | Concat- once | Concat-twice |
|---|
AA BB | AB AB | AB BA | AA BB | AB AB | AB BA |
|---|
| √ | | | | | | 89.4 | √ | | | | | | 87.2 | | √ | √ | | | | | 89.7 | √ | √ | | | | | 88.3 | | √ | √ | √ | | | | 89.7 | √ | √ | √ | | | | 88.8 | | √ | √ | | √ | | | 91.7 | √ | √ | | √ | | | 88.5 | | √ | √ | | | √ | | 90.5 | √ | √ | | | √ | | 89.0 | | √ | √ | | | | √ | 91.7 | √ | √ | | | | √ | 89.8 |
|
Table 3. Results of ablation experiment
| Model | Year | Accuracy /% | Parameter /106 |
|---|
| VGG-16 | 2015 | 82.6 | 134.29 | | VGG-19 | 2015 | 81.6 | 139.59 | | Inception-V2 | 2015 | 83.5 | 10.16 | | Inception-V3 | 2016 | 87.0 | 22.77 | | Inception-V4 | 2017 | 86.4 | 42.14 | | ResNet-50 | 2016 | 89.4 | 23.52 | | ResNet-101 | 2016 | 90.5 | 42.51 | | ResNet-152 | 2016 | 87.0 | 58.16 | | DenseNet-63 | 2017 | 87.2 | 2.31 | | DenseNet-121 | 2017 | 87.8 | 6.96 | | DenseNet-169 | 2017 | 87.1 | 12.49 | | DCNet-18 | 2017 | 87.0 | 41.93 | | DCNet-101 | 2017 | 85.9 | 42.58 | | NTS | 2018 | 84.0 | 26.25 | | DCL | 2019 | 86.6 | 23.52 | | HRNet | 2020 | 87.6 | 39.20 | | Proposed-DenseNet-63 | 2021 | 89.8 | 6.07 | | Proposed-ResNet-50 | 2021 | 91.7 | 27.28 |
|
Table 4. Comparison of recognition accuracy with popular CNN
| Place of training | Place of testing | ResNet-101 | Proposed-ResNet-50 |
|---|
| Changle， Jinan， Minhou， Minqing， Fuqing， Yongtai | Lianjiang， Luoyuan | 52.7 | 54.1 | | Changle， Jinan， Fuqing， Yongtai， Lianjiang， Luoyuan | Minhou， Minqing | 51.6 | 60.6 | | Changle， Jinan， Minhou， Minqing， Lianjiang， Luoyuan | Fuqing， Yongtai | 49.7 | 52.8 | | Minhou， Minqing， Fuqing， Yongtai， Lianjiang， Luoyuan | Changle， Jinan | 46.4 | 46.7 |
|
Table 5. Comparison of recognition accuracy with different locations between training set and test set
| Place | ResNet-101 | Proposed-ResNet-50 |
|---|
| Changle | 85.3 | 88.3 | | Fuqing | 86.8 | 87.3 | | Jinan | 82.4 | 83.0 | | Lianjiang | 85.8 | 89.5 | | Luoyuan | 83.1 | 84.7 | | Minhou | 86.2 | 90.2 | | Minqing | 89.9 | 91.4 | | Yongtai | 85.9 | 89.0 |
|
Table 6. Comparison of recognition accuracy with different locations
| Case | ResNet-101 | Proposed-ResNet-50 |
|---|
| Case 1 | 76.4 | 76.8 | | Case 2 | 75.6 | 77.4 | | Case 3 | 73.7 | 74.5 |
|
Table 7. Comparison of recognition accuracy after data balance
