• Laser & Optoelectronics Progress
  • Vol. 61, Issue 18, 1812004 (2024)
Ruanzhao Guo1, Ke Wang1, Huiqin Wang1,*, Zhan Wang2..., Gang Zhen2, Yuan Li3 and Jiachen Li1|Show fewer author(s)
Author Affiliations
  • 1School of Information and Control Engineering, Xi'an University of Architecture and Technology, Xi'an 710055, Shaanxi, China
  • 2Shaanxi Provincial Institute of Cultural Relics Protection, Xi'an 710075, Shaanxi, China
  • 3Xi'an Museum, Xi'an 710074, Shaanxi, China
  • show less
    DOI: 10.3788/LOP240448 Cite this Article Set citation alerts
    Ruanzhao Guo, Ke Wang, Huiqin Wang, Zhan Wang, Gang Zhen, Yuan Li, Jiachen Li. Anti-Disturbance Cross-Scene Multispectral Imaging Pigment Classification Method for Painted Cultural Relics[J]. Laser & Optoelectronics Progress, 2024, 61(18): 1812004 Copy Citation Text show less
    Schematic diagram of the process for anti-disturbance cross-scene multispectral imaging for classification of pigments in painted cultural relics
    Fig. 1. Schematic diagram of the process for anti-disturbance cross-scene multispectral imaging for classification of pigments in painted cultural relics
    Spectral image of the mural in the 660 nm channel
    Fig. 2. Spectral image of the mural in the 660 nm channel
    Flowchart of anti-spectral disturbance
    Fig. 3. Flowchart of anti-spectral disturbance
    Schematic diagram of the overlapping region of two shots. (a) Sketch of the mural sub-lens shooting; (b) multispectral images of overlapping region of sub-lens; (c) spectral image and grayscale histogram of 660 nm overlapping region
    Fig. 4. Schematic diagram of the overlapping region of two shots. (a) Sketch of the mural sub-lens shooting; (b) multispectral images of overlapping region of sub-lens; (c) spectral image and grayscale histogram of 660 nm overlapping region
    Basic image and histogram specification images. (a) Basic image Lij; (b) histogram specification image Nij
    Fig. 5. Basic image and histogram specification images. (a) Basic image Lij; (b) histogram specification image Nij
    Anti-spectral perturbation results. (a) Basic imag Sij; (b) anti-spectral perturbation image Fij; (c) Sij histogram;(d) Fij histogram;(e) two-shot multi-spectral image stitching after anti-spectral perturbation
    Fig. 6. Anti-spectral perturbation results. (a) Basic imag Sij; (b) anti-spectral perturbation image Fij; (c) Sij histogram;(d) Fij histogram;(e) two-shot multi-spectral image stitching after anti-spectral perturbation
    Painted cultural relic pigment classification network structure with cross-scene domain shift attributes
    Fig. 7. Painted cultural relic pigment classification network structure with cross-scene domain shift attributes
    Diagram of encoder
    Fig. 8. Diagram of encoder
    Structure diagram of feature extractor
    Fig. 9. Structure diagram of feature extractor
    Flowchart of classification and identification of pigments in painted cultural relics
    Fig. 10. Flowchart of classification and identification of pigments in painted cultural relics
    Schematic diagram of multispectral imaging system
    Fig. 11. Schematic diagram of multispectral imaging system
    Experimental data of anti-spectral perturbation by sub-shot. (a) Schematic diagram of shot shooting of simulated murals; (b) shot 1 captures multispectral images at 620 nm; (c) shot 2 captures multispectral images at 620 nm; (d) multispectral images before resisting spectral perturbation at 620 nm; (e) multispectral images after resisting spectral perturbation at 620 nm
    Fig. 12. Experimental data of anti-spectral perturbation by sub-shot. (a) Schematic diagram of shot shooting of simulated murals; (b) shot 1 captures multispectral images at 620 nm; (c) shot 2 captures multispectral images at 620 nm; (d) multispectral images before resisting spectral perturbation at 620 nm; (e) multispectral images after resisting spectral perturbation at 620 nm
    Ground truth and marking diagram of simulated mural. (a) Ground truth of simulated mural; (b) marking diagram of simulated mural
    Fig. 13. Ground truth and marking diagram of simulated mural. (a) Ground truth of simulated mural; (b) marking diagram of simulated mural
    OA and Kappa coefficients of each algorithm before and after anti-spectral disturbance
    Fig. 14. OA and Kappa coefficients of each algorithm before and after anti-spectral disturbance
    Images of simulated pigment plate and simulated mural at 620 nm. (a) Multispectral image of pigment plate at 620 nm; (b) interest marker image of pigment board points; (c) simulation mural of single exposure shot at 620 nm
    Fig. 15. Images of simulated pigment plate and simulated mural at 620 nm. (a) Multispectral image of pigment plate at 620 nm; (b) interest marker image of pigment board points; (c) simulation mural of single exposure shot at 620 nm
    Generalization ability experiment. (a) Generated data distribution; (b) data distribution after processing by the discriminator
    Fig. 16. Generalization ability experiment. (a) Generated data distribution; (b) data distribution after processing by the discriminator
    Results of cross-scene classification between paint panels and simulated murals. (a) Ground truth; (b) DHCNet[23]; (c) FDSSC[24]; (d) SSRN[25]; (e) SSKR[26]; (f) HPDM-SPRN[27]; (g) BASSNet[28]; (h) DFFN[29]; (i) proposed method
    Fig. 17. Results of cross-scene classification between paint panels and simulated murals. (a) Ground truth; (b) DHCNet[23]; (c) FDSSC[24]; (d) SSRN[25]; (e) SSKR[26]; (f) HPDM-SPRN[27]; (g) BASSNet[28]; (h) DFFN[29]; (i) proposed method
    Schematic diagram of the scene shooting of the Mingwang statue in Dule Temple
    Fig. 18. Schematic diagram of the scene shooting of the Mingwang statue in Dule Temple
    Colored and labeled images of mural. (a) Mural color image; (b) shot 1 training samples
    Fig. 19. Colored and labeled images of mural. (a) Mural color image; (b) shot 1 training samples
    Classification effect of different methods on murals. (a) Proposed method; (b) DHCNet[23]; (c) FDSSC[24]; (d) SSRN[25]; (e) SSKR[26]; (f) HPDM-SPRN[27]; (g) BASSNet[28]; (h) DFFN[29]; (i) SVM
    Fig. 20. Classification effect of different methods on murals. (a) Proposed method; (b) DHCNet[23]; (c) FDSSC[24]; (d) SSRN[25]; (e) SSKR[26]; (f) HPDM-SPRN[27]; (g) BASSNet[28]; (h) DFFN[29]; (i) SVM
    Schematic diagram of XRF test points and multispectral images of the Thirteenth Venerable
    Fig. 21. Schematic diagram of XRF test points and multispectral images of the Thirteenth Venerable
    XRF analysis results of the Thirteenth Venerable. (a) XRF analysis result of point 1; (b) XRF analysis result of point 2; (c) XRF analysis result of point 3; (d) XRF analysis result of point 4
    Fig. 22. XRF analysis results of the Thirteenth Venerable. (a) XRF analysis result of point 1; (b) XRF analysis result of point 2; (c) XRF analysis result of point 3; (d) XRF analysis result of point 4
    Pigment multispectral image training set
    Fig. 23. Pigment multispectral image training set
    Diagram of the pigment distribution of the Thirteenth Holiness
    Fig. 24. Diagram of the pigment distribution of the Thirteenth Holiness
    NumberCategoryNumber of samples
    0Background898151
    1Vermilion56752
    2Lazurite80086
    3Minium1167
    4Mineral green44533
    5Chrome yellow301516
    6Graphite49720
    Table 1. Simulated mural multi-spectral image sample
    AlgorithmHPDM-SPRN27BASSNet28DFFN29Proposed
    Before resisting spectral perturbation
    After resisting spectralperturbation
    Table 2. Classification results of different methods before and after anti-spectral perturbation
    ClassHPDM-SPRN27BASSNet28DFFN29Proposed
    Before calibrationAfter calibrationBefore calibrationAfter calibrationBefore calibrationAfter calibrationBefore calibrationAfter calibration
    Vermilion67.1298.0896.3094.1770.7597.9197.8899.8
    Lazurite97.9197.9299.0299.2998.3398.7596.7394.5
    Minium95.2097.5191.9594.1795.4697.3473.69100
    Mineral green94.3275.0068.7682.9783.5275.2957.9494.9
    Chrome yellow82.0491.2591.9095.6894.6993.9197.0099.3
    Graphite85.1388.0982.7781.5689.5883.7697.3890.9
    OA /%84.1791.3490.6593.6891.2892.5793.7897.49
    Kappa×10076.8086.7285.5890.1386.5288.4695.8398.30
    Table 3. Classification accuracy of different methods before and after anti-spectral disturbance
    AlgorithmRMSEPSNRSSIM
    Before calibrationAfter calibrationBefore calibrationAfter calibrationBefore calibrationAfter calibration
    DHCNet231.841.5722.4225.020.960.96
    FDSSC242.101.5621.8625.110.960.97
    SSRN251.851.9920.8522.950.960.96
    SSKR262.041.6520.4623.300.950.96
    HPDM-SPRN271.981.6820.7024.350.940.96
    BASSNet281.761.4524.6325.540.960.97
    DFFN291.491.5525.2324.650.970.97
    Proposed1.530.7027.5128.540.990.99
    Table 4. Quality objective evaluation results of image
    NumberCategoryNumber of samples(source domain)Number of samples(target domain)
    0Background396043939600
    1Vermilion2876356775
    2Lazurite2971579070
    3Minium127851270
    4Mineral green4681449590
    5Chrome yellow19805284795
    Table 5. Samples of simulated mural multispectral images
    ClassDHCNet23FDSSC24SSRN25SSKR26HPDM-SPRN27BASSNet28DFFN29Proposed
    Vermilion61.4299.9098.94010099.93089.66
    Lazurite10099.9898.630099.9610095.29
    Minium94.4110096.851.811.2657.3263.6298.11
    Mineral green73.5399.8199.8910016.0426.1446.1199.27
    Chrome yellow63.5470.5734.4942.730.2488.8582.7685.91
    OA /%70.5382.1860.0536.3313.8785.3671.7889.37
    Kappa×10056.2573.4148.6816.89-5.6575.8654.2979.02
    Table 6. Classification accuracy of different methods on pigment cross-scene
    AlgorithmRMSEPSNRSSIM
    DHCNet232.8821.960.95
    FDSSC242.0319.690.95
    SSRN253.0716.350.92
    SSKR265.0520.350.95
    HPDM-SPRN276.3319.270.95
    BASSNet282.4526.490.97
    DFFN293.2624.010.96
    Proposed1.41825.930.98
    Table 7. Results of objective evaluation of image quality cross-scene
    ModelOA /%Kappa×100
    Remove encoder82.6273.54
    Remove contrastive learning84.4074.82
    Remove adversarial learning86.3476.58
    Proposed89.3779.02
    Table 8. Ablation experiments of different modules
    Ruanzhao Guo, Ke Wang, Huiqin Wang, Zhan Wang, Gang Zhen, Yuan Li, Jiachen Li. Anti-Disturbance Cross-Scene Multispectral Imaging Pigment Classification Method for Painted Cultural Relics[J]. Laser & Optoelectronics Progress, 2024, 61(18): 1812004
    Download Citation