• Laser & Optoelectronics Progress
  • Vol. 59, Issue 24, 2411001 (2022)
Dongbin Liu1, Huiqin Wang1,*, Ke Wang1, Zhan Wang2, and Gang Zhen2
Author Affiliations
  • 1School of Information and Control Engineering, Xi'an University of Architecture and Technology, Xi'an 710055, Shaanxi, China
  • 2Shaanxi Provincial Institute of Cultural Relics Protection, Xi'an 710075, Shaanxi, China
  • show less
    DOI: 10.3788/LOP202259.2411001 Cite this Article Set citation alerts
    Dongbin Liu, Huiqin Wang, Ke Wang, Zhan Wang, Gang Zhen. Blind Restoration Method for Incomplete and Sparse Text Images Based on Content Style Transfer[J]. Laser & Optoelectronics Progress, 2022, 59(24): 2411001 Copy Citation Text show less
    Schematic of GAN
    Fig. 1. Schematic of GAN
    Structure diagram of blind restoration network for incomplete sparse text image
    Fig. 2. Structure diagram of blind restoration network for incomplete sparse text image
    Max pooling diagram
    Fig. 3. Max pooling diagram
    ISA mechanism diagram
    Fig. 4. ISA mechanism diagram
    Generator network structure based on improved self-attention
    Fig. 5. Generator network structure based on improved self-attention
    Discriminator network structure based on improved self-attention mechanism
    Fig. 6. Discriminator network structure based on improved self-attention mechanism
    Partial incomplete characters in Tianjin dule Temple
    Fig. 7. Partial incomplete characters in Tianjin dule Temple
    Segmentation results of incomplete text in Tianjin dule Temple
    Fig. 8. Segmentation results of incomplete text in Tianjin dule Temple
    Some regular script data of Ouyang xun, Wang Xizhi, Su Shi, Huang Tingjian, etc
    Fig. 9. Some regular script data of Ouyang xun, Wang Xizhi, Su Shi, Huang Tingjian, etc
    Comparison of experimental results. (a) Broken images; (b) CycleGAN; (c) CycleGAN (LS+SA); (d) proposed algorithm;(e) original images
    Fig. 10. Comparison of experimental results. (a) Broken images; (b) CycleGAN; (c) CycleGAN (LS+SA); (d) proposed algorithm;(e) original images
    “Xi” “Du” “Nian” and “Huai” calligraphy image AHE. (a) “Xi”calligraphy image AHE; (b) “Du” calligraphy image AHE; (c) “Nian” calligraphy image AHE; (d) “Huai” calligraphy image AHE
    Fig. 11. “Xi” “Du” “Nian” and “Huai” calligraphy image AHE. (a) “Xi”calligraphy image AHE; (b) “Du” calligraphy image AHE; (c) “Nian” calligraphy image AHE; (d) “Huai” calligraphy image AHE
    Image preprocessing results. (a) Original images; (b) AHE+SR; (c) text extraction
    Fig. 12. Image preprocessing results. (a) Original images; (b) AHE+SR; (c) text extraction
    Comparison results of different repair algorithms.(a) Adding mask artificially; (b) RFR; (c) proposed algorithm
    Fig. 13. Comparison results of different repair algorithms.(a) Adding mask artificially; (b) RFR; (c) proposed algorithm
    Evaluation indexAlgorithmCalligraphy image
    PSNRCycleGAN12.2911.9812.9914.0512.1412.4413.3113.32
    CycleGAN(LS+SA)12.6313.1613.2915.2312.1813.2113.5913.52
    Proposed algorithm13.2412.4114.0415.6512.7314.2714.5914.71
    SSIMCycleGAN0.75350.73740.78730.80380.72520.77950.81550.7854
    CycleGAN(LS+SA)0.74050.76690.78200.82290.73080.79330.81260.7817
    Proposed algorithm0.77200.73920.80140.83720.74570.81440.82580.8143
    RMSECycleGAN61.9264.2257.1450.6262.3160.8755.0655.03
    CycleGAN(LS+SA)59.5656.0255.2044.1462.6955.7353.3153.75
    Proposed algorithm55.5861.0850.6442.0558.8949.3047.5346.86
    Table 1. PSNR, SSIM, and RMSE comparison results
    DatasetCalligraphy image dataset
    Mask ratio /%
    11.5312.5423.7328.2732.1843.3954.9562.73
    PSNRCycleGAN13.9714.5313.5414.4314.1111.8511.5610.11
    CycleGAN(LS+SA)14.4015.1014.4714.5415.2012.4511.7411.16
    Proposed algorithm14.2013.7514.2914.6715.6313.0912.5811.81
    SSIMCycleGAN0.78760.79680.80090.79660.83950.75620.75640.6988
    CycleGAN(LS+SA)0.81030.83800.83020.81010.86390.76970.75090.7162
    Proposed algorithm0.81200.79900.82540.80660.84780.79210.77130.7409
    RMSECycleGAN51.1247.8753.6748.4250.2265.1467.3579.66
    CycleGAN(LA+SA)48.5844.8248.1847.8344.3160.8266.0270.51
    Proposed algorithm49.6952.3949.2447.1642.1956.5259.9065.49
    Table 2. Comparison results of PSNR, SSIM, and RMSE under various failure ratios
    ScoreObstruction criterionQuality standard
    5No deterioration in image qualityVery good
    4The image quality has deteriorated,not hinder viewingGood
    3The deterioration of image quality slightly hinders viewingGeneral
    2Obstruction to viewingBad
    1Very serious obstruction of viewingVery bad
    Table 3. CCIR500-1 Subjective evaluation scale
    Proposed resultRFR resultProposed scoreRFR score
    4.22.6
    3.91.7
    4.92.5
    3.82.0
    Table 4. Subjective evaluation results
    Dongbin Liu, Huiqin Wang, Ke Wang, Zhan Wang, Gang Zhen. Blind Restoration Method for Incomplete and Sparse Text Images Based on Content Style Transfer[J]. Laser & Optoelectronics Progress, 2022, 59(24): 2411001
    Download Citation