Using conventional spatial light modulators (SLMs) for complex modulation of coherent light beams is a challenging task. Because current SLMs cannot perform full complex modulation on a single panel, the complex holograms generated by computers must be converted into amplitude-only holograms or phase-only holograms^[1–3]. In 1970, Lee decomposed a two-dimensional complex field into two functions with a constant amplitude in conjunction with different spatially distributed phase functions^[4]. The complex field was then retrieved through the coherent superposition of the two waves. In 1978, Hsueh and Sawchuk used phase functions to generate double-phase holograms (DPHs) for binary devices^[5]. In these methods, two phase-only patterns are encoded into a phase-only hologram by using combined sub-cells to produce the desired complex field.

DPHs have attracted significant attention because they can be implemented using phase-only SLMs, such as liquid crystal SLMs^[6,7]. In 2002, Arrizón modified a double-phase holographic code to be used through phase-only SLMs without increasing the complexity of hologram cells^[8–10], since a pixelated complex field uses doubled or quadrupled number of phase-only pixels on the SLM, which can only modulate a complex hologram with a half or quarter of its resolution. In 2014, Mendoza-Yero et al. used binary gratings to sample phase elements and combined them with a $4\text{-}f$ optical system to synthesize complex fields^[11]. This method allows for single-pixel operation without an array of subpixels for codifying each pixel in an input plane. It maintains the same image size as that of the original image, but results in a sampling loss. Various attempts based on the single-pixel on-axis DPH configuration have been made to improve the image quality using blazed gratings^[12] and realize complex modulation for three-dimensional scenes^[13]. In 2019, a DPH method with a weight factor was introduced to reduce the peripheral noise and expand the space-bandwidth product (SBP) for DPHs^[14], in which the weight factor is multiplied by an amplitude and used to modify the phase function for reducing the noise in image peripherals and extending expressible areas.

In the traditional digital encoding methods, two phases can hardly be interweaved completely into a DPH, whose size is identical to that of an original complex field, without information loss. In this work, we propose a spatiotemporal encoded DPH to suppress information loss by using multiple resampled sub-DPHs together with time multiplexing, which can reduce speckles and expand viewing zones in holographic displays^[15–18]. Such a spatiotemporal multiplexing method combines spatial coding and temporal coding to achieve high-quality image reconstructions with the proposed DPHs, without reducing image sizes or losing any terms of complex fields.

From Fig. 1(a), it is clear that the ${(i,j)}^{\mathrm{th}}$ pixel value of a pixelated complex hologram $u(x,y)$ can be expressed by its amplitude and phase as ${u_h}^{i,j}=A^{i,j}\exp (i{\varphi }^{i,j})$, where $i$ and $j$ are the pixel indices. The pixels of the complex hologram $u_h(x,y)$ are then divided into four groups $u_1(x,y)$, $u_2(x,y)$, $u_3(x,y)$, and $u_4(x,y)$ consisting of non-adjacent pixels. The neighbors of every pixel in each group are excluded from the same group. Therefore, it is convenient to transform pixels in a group into macro-pixels formed by arrays of $2\times 2$ phase-only pixels, as is shown in Fig. 2.

We assume that the desired complex field is encoded by a phase-only SLM with rectangular pixels having a rectangular pixel-active window with dimensions of $a\times b$ and pixel distances of $\alpha$ and $\beta$. The rectangular functions that represent the SLM pixels and complex macro-pixels are defined as $${\omega }_h(x,y)=\mathrm{rect}(x/a)\mathrm{rect}(y/b),$$(1)$${\omega }_c(x,y)=\mathrm{rect}(x/2\alpha )\mathrm{rect}(y/2\beta ).$$(2)

The complex wavefront of the group $u_1(x,y)$ modulated by a spatially quantized element can be expressed as^[5]$$u_1(x,y)=\sum _{m,n}{u}_1^{m,n}{\omega }_c(x-2n\alpha ,y-2m\beta ),$$(3)where $u_1^{m,n}=A^{m,n}\exp (i{\varphi }^{m,n})$ is the ${(m,n)}^{\mathrm{th}}$ pixel value of $u_1(x,y)$. By assuming that $u$ and $v$ are spatial frequencies, the desired reconstruction of a complex hologram can be obtained by applying a Fourier transform and bandwidth limiting as $$U_1(u,v)=p(u,v)\sum _{m,n}{u}_1^{m,n}\exp [-i2\pi (2n\alpha u,2m\beta v)],$$(4)where $p(u,v)=4\alpha \beta \mathrm{sinc}(2\alpha u)\mathrm{sinc}(2\beta v)$ is the bandwidth-limited function.

To obtain the encoded complex field, every pixel in the complex hologram is transformed into a macro-pixel by using an array of $2\times 2$ phase-only pixels to perform complex decomposition [Fig. 1(b)]. The decomposed phase functions are $$h_1^{m,n}=\exp ({\varphi }^{m,n}-{\mathrm{\Delta }}^{m,n}),$$(5)$$h_2^{m,n}=\exp ({\varphi }^{m,n}+{\mathrm{\Delta }}^{m,n}),$$(6)where the phase shift is ${\mathrm{\Delta }}^{m,n}=\arccos (A^{m,n})$. Knowing that $h_{\mathrm{sub}1}(x,y)=\sum _{m,n}{h}^{m,n}(x,y)$, we refer to other transformed groups of pixels into sub-DPHs $h_{\mathrm{sub}2}(x,y)$, $h_{\mathrm{sub}3}(x,y)$, and $h_{\mathrm{sub}4}(x,y)$, each of which contains only 1/4 information in the complex hologram, but has the same size as the complex hologram.

The sub-DPHs are sequentially uploaded onto a phase-only SLM and reconstructed on the back focal plane of a $4\text{-}f$ system. The SLM should be equipped with a high frame rate to ensure that the sub-DPHs are played at an adequately short time interval, so that all the macro-pixels from sub-DPHs are combined for time-sequential display, which are captured by human eyes or detectors. Thus, the complex hologram can be fully reconstructed without down-sampling.

The reconstructed image from the sub-DPH $h_{\mathrm{sub}1}(x,y)$ encoded by the macro-pixels can be expressed by^[5]$$\begin{gathered} H_1(u,v)=p_S(u,v)\sum _{m,n}\{\cos ({\mathrm{\Delta }}^{m,n})\exp (i{\varphi }^{m,n}) \\ ·\exp [-i2\pi (2n\alpha u+2m\beta v)]\} \\ +p_N(u,v)\sum _{m,n}\{\sin ({\mathrm{\Delta }}^{m,n})\exp (i{\varphi }^{m,n}) \\ ·\exp [-i2\pi (2n\alpha u+2m\beta v)]\}, \end{gathered}$$(7)where $p_S(u,v)$ and $p_N(u,v)$ donate the following bandwidth-limited functions: $$p_S(u,v)=4ab\cos (\pi \alpha u)·\cos (\pi \beta v)\mathrm{sinc}(au)\mathrm{sinc}(bv),$$(8)$$p_N(u,v)=-i4ab\sin (\pi \alpha u)·\sin (\pi \beta v)\mathrm{sinc}(au)\mathrm{sinc}(bv).$$(9)$H_1(u,v)$ consists of a signal term, which is identical with that of the original field, and a noise term generated by the spatial shifts between the two different phases that encode the complex field. The quality of reconstruction largely depends on the bandwidth-limited functions $p_S(u,v)$ and $p_N(u,v)$. The cosine factors of signal function $p_S(u,v)$ restrict the hologram signal field to the on-axis centered square band with dimensions of $\mathrm{\Delta }u=1/\alpha$ and $\mathrm{\Delta }v=1/\beta$. The noise function $p_N(u,v)$ has a vertical profile that is null along the axes of $u=0$ and $v=0$. It should be noted that the maximum value of $p_N(u,v)$ occurs along the axes of $u=v$ and $u=-v$. To block high-noise areas in the spectrum plane, a filter is introduced usually in reconstruction.

But, the interweaving of two functions makes it unavoidable to allow part of the noise to pass through and lose part of the signal, resulting in reconstruction noise or signal loss. Since the factor $\sin ({\mathrm{\Delta }}^{m,n})\exp (i{\varphi }^{m,n})$ varies with different sub-DPHs, impacts of the noise term are weakened by time-averaging of four sub-DPHs. A bandlimit by the requisite filter in the Fourier plane is eased, and the available bandwidth is thus expanded.

To illustrate the working principles of the proposed spatiotemporal multiplexing method, two test images with the same constant phase but different amplitudes are considered. One has mostly low-spatial-frequency components [Fig. 3(a)] and the other has mostly high-spatial-frequency components [Fig. 3(e)]. Each image was encoded using a single-pixel DPH, four separated sub-DPHs, and a spatiotemporal multiplexing DPH, respectively. The images reconstructed from single-pixel DPH [Figs. 3(b) and 3(f)] contain ground noises up to 79.6 in gray value. The standard deviation for the high-intensity section of Fig. 3(b) is 11.36 in gray value, and that of Fig. 3(f) is 37.73 in gray value. They have inaccurate edges and details, indicating the loss of high-spatial-frequency components. In the reconstructions based on sub-DPHs [Figs. 3(c) and 3(g)], the ground noise is alleviated by macro-pixel encoding to almost null. In the high-intensity sections of the images, there are major distortions, representing complementary shapes among the four reconstructions. When time encoding is introduced to coordinate macro-pixel encoding, intensity multiplication enhances the contrast in the image to an accurate level. The standard deviation for the high-intensity section of Fig. 3(d) is 7.10 in gray value, and that of Fig. 3(h) is 32.57 in gray value.

Figure 4 presents the numerical reconstruction of a complex hologram with an image size of $512\times 512\mathrm{pixels}$, which is generated from an original image in the University of Southern California-Signal and Image Processing Institute (USC-SIPI) Image Database^[19]. The image reconstructed by the spatiotemporal multiplexing DPH method [Fig. 4(d)] includes more details and less noise than the image reconstructed by the single-pixel DPH method [Fig. 4(b)]. It also has a much higher peak signal-to-noise ratio (PSNR) of 21.21 dB and structure similarity index (SSIM) of 0.59, while the single-pixel method yields an image with a PSNR of 18.50 dB and SSIM of 0.33. It indicates a 14.64% increase in PSNR and a 78.79% increase in SSIM, respectively. The parameter of the filter has impacts on the image quality. Curves of PSNRs and SSIMs for the reconstructed images changing with the diameter of filter are provided in Figs. 4(e) and 4(f), respectively. These improvements show that the image reconstructed by the spatiotemporal multiplexing DPH method is much closer to the original image in terms of very minute details. The image reconstructed by one of the four sub-DPHs without time encoding [Fig. 4(c)] also shows improved performance in terms of noise suppression compared to the single-pixel method.

To demonstrate the effectiveness of the proposed spatiotemporal multiplexing method experimentally, we implemented the optical setup, as shown in Fig. 5. A coherent beam at $\lambda =532\mathrm{nm}$ is emitted from a solid-state laser acting as a light source. It is then attenuated and expanded before passing through a polarizer and onto a reflective liquid crystal on silicon (LCoS) phase-only SLM (Holoeye Gaya). The SLM has a pixel number of $3840\times 2160$, pixel pitch of 3.74 μm, and frame rate of 60 Hz. The desired complex hologram is decomposed and encoded into four sub-DPHs, $h_{\mathrm{sub}1}(x,y)$, $h_{\mathrm{sub}2}(x,y)$, $h_{\mathrm{sub}3}(x,y)$, and $h_{\mathrm{sub}4}(x,y)$, which are then time-sequentially uploaded onto the SLM at a rate of 60 Hz. The beam splitter (BS) allows for both the plane wave illuminating onto the SLM as well as the reflection towards a $4\text{-}f$ system with a filter to block unwanted diffraction orders.

Figure 6 shows the optically reconstructed images captured by using a complementary metal–oxide–semiconductor (CMOS) detector. It can be seen that the image reconstructed by the spatiotemporal multiplexing DPH method [Fig. 6(d)] retains most of its original features. Compared to the single-pixel method [Fig. 6(b)], it preserves the edges and most of the details of the image with much less blurring and noise. It has the PSNR of 11.44 dB and SSIM of 0.18, while the image reconstructed by the single-pixel method has only PSNR of 11.30 dB and SSIM of 0.14, respectively.

In summary, the proposed spatiotemporal multiplexing DPH method can deliver high-quality images through SBP-preserved resampling and increased time-bandwidth product. It improves the capability of DPHs to preserve the details of reconstructed images and suppresses the information loss during hologram interweaving. This makes the spatiotemporal DPHs a suitable method for the digital modulation of both static and quasi-static complex fields with existing SLMs. It can be used in a wide range of applications based on complex holograms, from high-quality holographic display to optical field generation.