• Acta Optica Sinica
  • Vol. 41, Issue 23, 2301003 (2021)
Xi Gong1, Zhanlong Chen1,2, Liang Wu1,2, Zhong Xie1,2,*, and Yongyang Xu1,2
Author Affiliations
  • 1School of Geography and Information Engineering, China University of Geosciences, Wuhan, Hubei 430074, China
  • 2National Engineering Research Center of Geographic Information System, Wuhan, Hubei 430074, China
  • show less
    DOI: 10.3788/AOS202141.2301003 Cite this Article Set citation alerts
    Xi Gong, Zhanlong Chen, Liang Wu, Zhong Xie, Yongyang Xu. Transfer Learning Based Mixture of Experts Classification Model for High-Resolution Remote Sensing Scene Classification[J]. Acta Optica Sinica, 2021, 41(23): 2301003 Copy Citation Text show less
    Flow chart of TLMoE
    Fig. 1. Flow chart of TLMoE
    Transfer learning process of expert network
    Fig. 2. Transfer learning process of expert network
    Training sample filter for expert networks
    Fig. 3. Training sample filter for expert networks
    Image examples of remote sensing scenes. (a) UCM dataset; (b) SIRI dataset; (c) RSSCN7 dataset
    Fig. 4. Image examples of remote sensing scenes. (a) UCM dataset; (b) SIRI dataset; (c) RSSCN7 dataset
    Classification confusion matrix of TLMoE-VGG19 on UCM dataset
    Fig. 5. Classification confusion matrix of TLMoE-VGG19 on UCM dataset
    Classification confusion matrix of TLMoE-VGG19 on SIRI dataset
    Fig. 6. Classification confusion matrix of TLMoE-VGG19 on SIRI dataset
    Classification confusion matrix of TLMoE-VGG19 on RSSCN7 dataset
    Fig. 7. Classification confusion matrix of TLMoE-VGG19 on RSSCN7 dataset
    Time consumption comparison before and after the combination of channels and pre-trained CNN in TLMoE. (a) VGG19; (b) Resnet50
    Fig. 8. Time consumption comparison before and after the combination of channels and pre-trained CNN in TLMoE. (a) VGG19; (b) Resnet50
    Comparison of different features on the 3 datasets by 2-dimensional feature visualization
    Fig. 9. Comparison of different features on the 3 datasets by 2-dimensional feature visualization
    No.Layer groupVGG19Resnet50
    Layer numberFeature sizeLayer numberFeature size
    1conv1264×224×224164×112×112
    2conv22128×112×1129256×56×56
    3conv34256×56×5612512×28×28
    4conv44512×28×28181024×14×14
    5conv54512×14×1492048×7×7
    6FC/GAP2409612048
    7output FC1100011000
    Table 1. Structure comparison between VGG19 and Resnet50
    No.MethodAccuracy /%
    1RF[20]44.77
    2SIFT+BoVW[21]76.81
    3SIFT+SPMK[22]75.29
    4VGG19 (training from scratch)83.48
    5Resnet50 (training from scratch)85.71
    6DCT-CNN[1]95.76
    7Pre-trained VGG19 features+SVM94.29
    8Pre-trained Resnet50 features+SVM97.14
    9GLDFB[5]97.62
    10TLMoE-VGG1998.10
    11TLMoE-Resnet5098.33
    Table 2. Classification accuracy comparison on the UCM dataset
    TypeRoad typeBuilding typeOther
    ClassFreewayIntersectionOverpassBuildingsLenseresidentialMediumresidentialTenniscourt
    GLDFB(VGG19)1001009590959590
    TLMoE-VGG19100100100909510095
    Table 3. Classification accuracy comparison on the confusing classes of UCM datasetunit: %
    No.MethodAccuracy /%
    1RF[20]49.90
    2SIFT+BoVW[21]75.63
    3SIFT+SPMK[22]77.69±1.01
    4VGG19 (training from scratch)86.13
    5Resnet50 (training from scratch)89.26
    6MCNN[23]93.75±1.13
    7Pre-trained VGG19 features+SVM94.79
    8Pre-trained Resnet50 features+SVM96.25
    9GLDFB[5]96.67
    10TLMoE-VGG1997.29
    11TLMoE-Resnet5097.50
    Table 4. Classification accuracy comparison on the SIRI dataset
    No.MethodAccuracy /%
    1RF[20]55.43
    2VGG19 (training from scratch)82.50
    3Resnet50 (training from scratch)81.70
    4Deep filter bank[24]90.04±0.6
    5Pre-trained VGG19 features+SVM91.93
    6Pre-trained Resnet50 features+SVM89.92
    7TLMoE-VGG1993.21
    8TLMoE-Resnet5093.29
    Table 5. Classification accuracy comparison on the RSSCN7 dataset
    No.ChannelAccuracy (pre-trained VGG19) /%Accuracy (pre-trained Resnet50) /%
    UCMSIRIRSSCN7UCMSIRIRSSCN7
    1Pre-judged channel94.6093.1387.5097.6296.2589.21
    2Expert channel93.8196.0492.1496.6797.0892.93
    3(FL(s), X(s))-SVM94.2994.7991.9397.1496.2589.92
    4TLMoE98.1097.2993.2198.3397.5093.29
    Table 6. Classification accuracy comparison between TLMoE channels
    No.FeatureAccuracy /%
    UCMSIRIRSSCN7
    1HOG52.1444.7935.79
    2SIFT58.3353.9654.14
    3LBP31.4346.2556.14
    4X(s)-VGG1993.3394.3890.71
    5Expert channel-VGG1993.8196.0492.14
    6X(s)-Resnet5095.4896.4692.14
    7Expert channel-Resnet5096.6797.0892.93
    Table 7. Classification accuracy comparison of several kinds of features
    Xi Gong, Zhanlong Chen, Liang Wu, Zhong Xie, Yongyang Xu. Transfer Learning Based Mixture of Experts Classification Model for High-Resolution Remote Sensing Scene Classification[J]. Acta Optica Sinica, 2021, 41(23): 2301003
    Download Citation