• Optics and Precision Engineering
  • Vol. 32, Issue 4, 549 (2024)
Jiajun ZHANG1, Jing LIAN1,2,*, Jizhao LIU2, Zilong DONG1, and Huaikun ZHANG2
Author Affiliations
  • 1School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou730000, China
  • 2School of Information Science and Engineering, Lanzhou University, Lanzhou730000, China
  • show less
    DOI: 10.37188/OPE.20243204.0549 Cite this Article
    Jiajun ZHANG, Jing LIAN, Jizhao LIU, Zilong DONG, Huaikun ZHANG. Using image smoothing structure information to guide image inpainting[J]. Optics and Precision Engineering, 2024, 32(4): 549 Copy Citation Text show less

    Abstract

    Using image structure features for image inpainting is a new method that has emerged in recent years with the widespread application of deep learning techniques. This method can generate plausible content within missing areas, but the restoration results heavily rely on the extracted content of image structures. In practical training, errors can propagate and accumulate, directly impacting the quality of the generated image when there is noise or distortion in the image structure. This method is still in the exploratory phase and faces challenges such as difficulty in network training, poor robustness, and inconsistent semantic context in generated images.To address these issues, this paper proposed a parallel network structure for image inpainting guided by smooth image structures. The generated content of the smooth image structure was not directly used as input for the next-level network but served as guidance information for the decoding layer of network. Additionally, to better match and balance the feature relationship between the structure and the image, this paper combined transformer and introduces a multi-scale feature guidence module. This module utilized the powerful modeling capability of transformers to establish connections between global features, matching and balancing features between structure and image textures.Experimental results demonstrate that the proposed method effectively restores missing content in images on three commonly used datasets and can be used as an image editing tool for object removal.
    Jiajun ZHANG, Jing LIAN, Jizhao LIU, Zilong DONG, Huaikun ZHANG. Using image smoothing structure information to guide image inpainting[J]. Optics and Precision Engineering, 2024, 32(4): 549
    Download Citation