• Optics and Precision Engineering
  • Vol. 31, Issue 16, 2430 (2023)
Sen WANG, Yang ZHU, Yinhui ZHANG*, Qingjian WANG, and Zifen HE
Author Affiliations
  • Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, China
  • show less
    DOI: 10.37188/OPE.20233116.2430 Cite this Article
    Sen WANG, Yang ZHU, Yinhui ZHANG, Qingjian WANG, Zifen HE. Multi-stage frame alignment video super- resolution network[J]. Optics and Precision Engineering, 2023, 31(16): 2430 Copy Citation Text show less

    Abstract

    Video-Super Resolution (VSR) aims to reconstruct low-resolution video frame sequences into high-resolution video frame sequences. Compared with single image super-resolution, VSR usually relies on the height-dependent information of neighboring frames to reconstruct the current frame because of the added information of temporal dimension. How to align adjacent frames and obtain highly correlated information between frames is the key issue of VSR task. In this paper, the VSR task is divided into three stages: deblurring, alignment, and reconstruction. In the deblurring stage, the current frame is pre-aligned with adjacent frames to obtain feature information highly related to the current frame, and the details of the current frame are enhanced to achieve more feature information extraction in the initial stage. In the alignment stage, the highly correlated information in adjacent frames is used to further strengthen the feature information in the current frame by performing a secondary alignment operation on the input features. In the reconstruction stage, raw low-resolution frames are aggregated to provide more feature information at the end of the network. In this paper, we use Multi-Layer Perceptron (MLP) instead of the traditional convolution operation to construct a feature extraction module, and also perform a secondary alignment of the generated feature information to refine the image features to obtain better video frame reconstruction results. The experimental results show that the proposed algorithm achieves a higher accuracy of video frame sequence reconstruction on a variety of publicly available datasets while achieving a lower number of network parameters and a more coherent video sequence reconstruction performance.
    Sen WANG, Yang ZHU, Yinhui ZHANG, Qingjian WANG, Zifen HE. Multi-stage frame alignment video super- resolution network[J]. Optics and Precision Engineering, 2023, 31(16): 2430
    Download Citation