Single-pixel imaging (SPI)^[1,2] is an emerging computational imaging method that collects the light intensity transmitted or reflected from the object in time sequence. The low manufacturing cost of the single-pixel detector provides SPI with the potential for various imaging applications including 3D imaging^[3–5], terahertz imaging^[6,7], diffraction and coherent imaging^[8,9], infrared imaging^[10,11], scattering medium imaging^[12,13], and single-photon imaging^[14,15]. Inspired by ghost imaging, SPI methods based on random illumination patterns were initially developed. In these methods, a large number of measurements should be acquired for high-fidelity imaging, but the time-consuming measurement acquisition and high computational burden limit the widespread applications of SPI.

To reduce the number of measurements and the computational cost, SPI methods based on orthogonal transformation have been proposed, such as Fourier single-pixel imaging (FSPI)^[16], discrete cosine transform SPI^[17], and discrete W transform SPI^[18]. Due to its excellent information compression capability, FSPI achieves high-quality reconstruction with fewer measurements than random illumination pattern-based SPI. In FSPI, the inverse Fourier transform (IFT) is employed to recover the images, so the computational burden can be significantly reduced. As a robust method, four-step phase-shift FSPI requires measurements twice the number of pixels of the basis pattern under full sampling. Three-step FSPI and two-step phase-shift FSPI^[19,20] were proposed to reduce the number of measurements but at the expense of noise immunity. Recently, FSPI based on deep learning and iteration^[21–24] was developed to decrease the number of measurements, but these methods still rely on four-step and three-step phase-shift patterns.

In this Letter, a robust two-step FSPI method is proposed to reduce the number of measurements without additional costs. In the proposed method, only two sets of phase-shift patterns (0 and $\pi /2$) are generated to acquire the corresponding measurements. The measurements from the base patterns are replaced by the average values of measurements from two sets of phase-shift patterns. Compared with the traditional two-step phase-shift FSPI method, our proposed method avoids the imaging degradation caused by the noise in the measurements from base patterns. Therefore, more reliable Fourier coefficients can be obtained without introducing additional patterns or measurements. IFT is conducted on these reliable Fourier coefficients to obtain high-quality reconstructed images. The simulations of different FSPI methods are carried out under noise and noise-free conditions, and experiments under strong light and weak light are also implemented. The results show that our method achieves similar image quality as four-step FSPI and three-step FSPI with much fewer measurements and more robust results than two-step FSPI.

The two-dimensional Fourier transform (FT) of a $M\text{pixel}\times N\text{pixel}$ image $O(x,y)$ is defined as $$\begin{gathered} O_{\mathrm{F}}(u,v)=F\{O(x,y)\} \\ =\sum _{x=1}^M\sum _{y=1}^{N}O(x,y)\exp [-\mathrm{j}2\pi (\frac{ux}{M}+\frac{vy}{N})], \end{gathered}$$(1)where $F\{·\}$ represents the FT, $(x,y)$ is the coordinate in the spatial domain, and $(u,v)$ is the coordinate in the Fourier domain.

Fourier basis pattern $P_{\phi }(x,y;u,v)$ can be obtained by applying an IFT to a delta function ${\delta }_F(u,v,\phi )$^[19] as $$P_{\phi }(x,y;u,v)=0.5+0.5·F^{-1}\{{\delta }_F(u,v,\phi )\},$$(2)where $F^{-1}\{·\}$ denotes an IFT, $\phi$ is the initial phase, and $${\delta }_F(u,v,\phi )=\{\begin{array}{cc}\exp (\mathrm{j}\phi ),& u=u_0,v=v_0\\ 0,& \text{other}\end{array}.$$(3)

Two patterns are used to acquire one spectral coefficient, that are $P_0(x,y;u,v)$ and $P_{\pi /2}(x,y;u,v)$, The corresponding measurement values are $D_0$ and $D_{\pi /2}$. They can be expressed as $$D_{\phi }(u,v)=\sum _{x=1}^M\sum _{y=1}^{N}O(x,y)·P_{\phi }(x,y;u,v)+N,$$(4)where $N$ represents the time-varying environmental noise.

For the traditional two-step phase-shift FSPI method, the Fourier spectral coefficient $F(u,v)$ is obtained by $$F(u,v)=(D_0-D_{\mathrm{base}})+\mathrm{j}(D_{\mathrm{base}}-D_{\pi /2}),$$(5)where $D_0$, $D_{\pi /2}$, and $D_{\mathrm{base}}$ are the measurement values corresponding to the Fourier basis patterns $P_0$, $P_{\pi /2}$, and base pattern $P_{\mathrm{base}}$. All the values of $P_{\mathrm{base}}$ are set as 126 in the 8-bit condition. Deng et al. projected the base pattern onto the objects 10 times to reduce the noise in measurements^[20]. The imaging quality of this method with gray patterns could be improved by sampling more times for each pattern, but it is not effective for binary patterns in noisy scenarios^[25].

To solve this problem, we replace $D_{\mathrm{base}}$ with the average values of $D_0$ and $D_{\pi /2}$ as follows: $$F(u,v)=(D_0-{\overline{D}}_0)+\mathrm{j}({\overline{D}}_{\pi /2}-D_{\pi /2}),$$(6)where $$\{\begin{array}{c}{\overline{D}}_0=\sum _{u=1}^M\sum _{v=1}^N{D}_0(u,v)/K-1\\ {\overline{D}}_{\pi /2}=\sum _{u=1}^M\sum _{v=1}^N{D}_{\pi /2}(u,v)/K\end{array},$$(7)represent the average values of $D_0$ and $D_{\pi /2}$. $K$ is the number of measurements. Note that the sum of $D_0(u,v)$ does not include the maximum measurement value. Since the number of measurements to obtain ${\overline{D}}_0$ and ${\overline{D}}_{\pi /2}$ is much larger than the number of measurements to obtain $D_{\mathrm{base}}$, this replacement could eliminate the time-varying noise more effectively and does not introduce additional patterns.

Finally, the reconstructed image $I_O$ can be obtained by conducting an IFT: $$I_O=F^{-1}\{F(u,v)\}.$$(8)

The procedure of the proposed method can be summarized in Fig. 1.

To test the imaging performance of the proposed method, four-step phase-shift FSPI^[16], three-step phase-shift FSPI^[19], and two-step phase-shift FSPI^[20] are compared with our proposed method. All these methods implement the IFT to reconstruct the image. Under-sampling simulations are performed on the Cameraman image with a resolution of $128\times 128$. For four-step phase-shift FSPI, three-step phase-shift FSPI, and two-step phase-shift FSPI, the numbers of measurements at the full sampling rate are 32,768, 24,576, and 16,394 (10 more base patterns), respectively. For our proposed method, the number of measurements at the full sampling rate is 16,384. The peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are the indicators to evaluate the imaging quality. High PSNR and SSIM values indicate better imaging results.

First, gray Fourier basis patterns are used for simulation. The sampling rates are set to 5%, 10%, 20%, and 30%, respectively. The simulation results are shown in Fig. 2 and Table 1. From the comparison of quantitative results, the PSNR values and the SSIM values of recovered images by four-step FSPI, three-step FSPI, and two-step FSPI are totally the same. The quantitative results of recovered images by our method are nearly the same as those of the other three methods.

Then, gray Fourier basis patterns are converted to binary patterns using spatial dither strategy^[26] for simulation. The simulation results are shown in Fig. 3 and Table 2. Because of the quantization errors and reduction in the imaging spatial resolution caused by spatial dithering strategy, the imaging quality is worse than gray patterns-based results. As the sampling rates increase, the image quality suffers more. The imaging quality of the two-step FSPI is worse than that of the other three methods because its base pattern is not applicable after spatial dithering^[25]. At the sampling rates of 20% and 30%, the image quality has a poor contrast. From the comparison of quantitative results, the PSNR values and the SSIM values of recovered images by four-step FSPI, three-step FSPI, and our method are nearly the same.

To test the noise robustness of the proposed method, we add white Gaussian noise to the raw data. The addition of white Gaussian noise is implemented using the built-in function awgn of MATLAB. All grayscale patterns are converted to binary patterns using the spatial dither strategy.

In the first simulation, the noise is set to $-20\mathrm{dB}$, and the sampling rates are set to 5%, 10%, 20%, and 30%, respectively. The simulation results are shown in Fig. 4 and Table 3. From the comparison of quantitative results, the PSNR values and the SSIM values of recovered images by four-step FSPI are basically the highest. The results of three-step FSPI are slightly worse than those of four-step FSPI. The PSNR results of our method are basically the same as that of four-step FSPI, and the SSIM results are slightly worse than those of four-step FSPI and three-step FSPI. The recovered image quality of two-step FSPI is obviously worse than that of our method due to poor noise robustness.

In the second simulation, the sampling rate of these methods is set to 20%, and the noises are set to $-10$, $-20$, $-30$, and $-40\mathrm{dB}$, respectively. The simulation results are shown in Fig. 5 and Table 4. From the comparison of quantitative results, the PSNR values and the SSIM values of recovered images by four-step FSPI are the highest. The results of three-step FSPI are slightly worse than those of four-step FSPI, especially when the noise level is high. The noise robustness of two-step FSPI is obviously worse than that of other methods. In contrast, our method has good noise robustness and can reach the level of four-step FSPI and three-step FSPI in most conditions.

To compare the performance of different methods with experiments, we establish an experimental system. The experimental system is shown in Fig. 6, using a white LED as the light source and a printed USAF resolution image with a toy Doraemon as the imaging object. The generated patterns are loaded by a digital micro-mirror device (DMD) (ViALUX V-9601, $1920\times 1200$), and the refresh rate of the DMD is set to 16,384 Hz. Since the resolution of the DMD is $1920\times 1200$, a series of $128\text{pixel}\times 128\text{pixel}$ patterns are upsampled to form $1024\text{pixel}\times 1024\text{pixel}$ binary patterns. The modulated light intensity signal is received by a photodetector (Thorlabs PDA100A2), and the gain is set to 50 dB. A data acquisition card (DAQ, NI USB-6366) converts the light signal into a digital signal that can be processed by a computer. Using the experimental system described above, four-step FSPI, three-step FSPI, and two-step FSPI are compared with our method at sampling rates of 5%, 10%, 20%, and 30%.

First, we use strong light to illuminate the target, and the maximum measurement value is $\sim 0.9\mathrm{V}$. The image reconstruction results are shown in Fig. 7. When the sampling rate is low, four-step FSPI and three-step FSPI can recover the images. However, at 20% and 30% sampling rates, both methods are influenced by environmental noise. The reconstructed images degrade and have low contrast. The performance of three-step FSPI is better than that of four-step FSPI, which is different from the simulation results. The results show that there are more complex factors in the actual environment. For two-step FSPI, the reconstructed images show artifacts in the four corners of the images even at a 5% sampling rate. When the sampling rate increases, the contrast of reconstructed images degrades quickly because of high noise. In contrast to the other three methods, our proposed method can obtain high-quality images at all four sampling rates.

In the second experiment, the light is weak to simulate a noisier environment. The maximum measurement value is $\sim 0.45\mathrm{V}$. The image reconstruction results are shown in Fig. 8. At four sampling rates, four-step FSPI and three-step FSPI reconstruct high-quality images. The results of three-step FSPI are slightly better than those of four-step FSPI. For two-step FSPI, the reconstructed images show artifacts in the four corners of the images as well. The contrast of reconstructed images deteriorates as the sampling rate increases, indicating low noise robustness of the method. For our proposed method, the reconstructed results are similar to those of four-step FSPI and three-step FSPI, which shows that the proposed method has good noise robustness and efficiency in the actual SPI system.

In this Letter, a robust two-step FSPI method is proposed. The measurements from base patterns are replaced by the average values of the measurements from two sets of phase-shift patterns. Therefore, our proposed method avoids the imaging degradation caused by the noise in the measurements from base patterns. Compared with four-step FSPI and three-step FSPI, our method achieves similar image quality with much fewer measurements. Compared with two-step FSPI, our method achieves more robust results. The proposed method provides an alternative approach for more efficient SPI.