Fringe projection profilometry (FPP) is one of the widely used structured light three-dimensional (3D) reconstruction methods^[1–4]. For various common highly reflective objects, the local intensity saturation of the captured fringe pattern leads to coding distortion, which makes it difficult for FPP to correctly extract the phase information^[5]. A large number of state-of-the-art methods have been proposed to address the above issue; these methods are called high-dynamic range (HDR) techniques by scholars. Commonly used HDR techniques include the following three categories: (a) Methods based on multiple exposure adjustment^[6–8]. Zhang and Yau^[6] first proposed the exposure time adjustment method. These methods usually have a good image signal-to-noise ratio (SNR); however, the exposure time range is difficult to quantify, and the large number of fringes sacrifices the measurement efficiency. (b) Methods based on adaptive fringe projection^[9–11]. These methods can adjust the pixel intensity of the projected fringe pattern according to the surface reflectivity. However, when the projected intensity decreases, the SNR of the captured fringe pattern will decrease as well. Further, for strongly reflective surfaces, adjusting pixel intensity is often not enough. (c) Methods based on polarizing filters^[12]. Specular reflected light is polarized, so this principle can be used to eliminate fringe saturation. However, it requires additional hardware and complex adjustment on site, and reduces the SNR in dark areas.

In this work, we introduce a $2f_H+a+2f_M+2f_L$ method for unwrapped phase extraction. For 3D reconstruction of non-diffuse surfaces, we introduce a new set of $\pi$-phase-shifted $2f_H+a/2+2f_M+2f_L$ fringe patterns with halved background intensity. If a pixel in the original fringe pattern is saturated, the corresponding pixel from the $\pi$-phase-shifted fringe pattern is selected for replacement. Eight replacement methods and their general formulas are given in detail.

Zhang et al.^[13] proposed a two-plus-one phase-shifting method, which can reduce the motion-induced phase error. Based on Zhang’s algorithm, we introduced a $2f_H+a+2f_M+2f_L$ algorithm (two high-frequency fringes, a background image, two mid-frequency fringes, and two low-frequency fringes) to calculate the unwrapped phase. For temporal phase unwrapping using the multifrequency method in this work, $f_L=1/W$, and $W$ is width of the projected fringe pattern. The captured fringe patterns can be expressed as $$\{\begin{array}{l}{g}_1(x,y)=a(x,y)+b(x,y)\sin {\phi }_H(x,y)\hfill \\ g_2(x,y)=a(x,y)\hfill \\ g_3(x,y)=a(x,y)+b(x,y)\cos {\phi }_H(x,y)\hfill \\ g_4(x,y)=a(x,y)+b(x,y)\sin {\phi }_M(x,y)\hfill \\ g_5(x,y)=a(x,y)+b(x,y)\cos {\phi }_M(x,y)\hfill \\ g_6(x,y)=a(x,y)+b(x,y)\sin {\phi }_L(x,y)\hfill \\ g_7(x,y)=a(x,y)+b(x,y)\cos {\phi }_L(x,y)\hfill \end{array},$$(1)where $a$ and $b$ represent the background and modulation intensities of the captured fringe patterns.

The wrapped phases can be expressed as $$\{\begin{array}{l}{\psi }_H(x,y)=\arctan \left[\frac{g_1(x,y)-g_2(x,y)}{g_3(x,y)-g_2(x,y)}\right]\hfill \\ {\psi }_M(x,y)=\arctan \left[\frac{g_4(x,y)-g_2(x,y)}{g_5(x,y)-g_2(x,y)}\right]\hfill \\ {\psi }_L(x,y)=\arctan \left[\frac{g_6(x,y)-g_2(x,y)}{g_7(x,y)-g_2(x,y)}\right]\hfill \end{array}.$$(2)

Since $f_L=1/W$, the unwrapped phase ${\phi }_L$ is equal to the wrapped phase ${\psi }_L$. The final unwrapped phase ${\phi }_H$ can be expressed as^[14]$$\{\begin{array}{l}{\phi }_L(x,y)={\psi }_L(x,y)\hfill \\ {\phi }_M(x,y)={\psi }_M(x,y)+2\pi \times \mathrm{round}\left[\frac{(f_M/f_L){\phi }_L(x,y)-{\psi }_M(x,y)}{2\pi }\right]\hfill \\ {\phi }_H(x,y)={\psi }_H(x,y)+2\pi \times \mathrm{round}\left[\frac{(f_H/f_M){\phi }_M(x,y)-{\psi }_H(x,y)}{2\pi }\right]\hfill \end{array},$$(3)where “round” is the rounding function.

To accurately calculate the three wrapped and unwrapped phases, none of the seven fringes in Eq. (1) can saturate. For a pixel at a certain index position, if any of the seven patterns saturates, it will cause a phase error. To this end, a $\pi$-phase-shifted $2f_H+a/2+2f_M+2f_L$ method is introduced whose fringe patterns are described as^[15]$$\{\begin{array}{l}{g}_1^{\prime }(x,y)=a(x,y)-b(x,y)\sin {\phi }_H(x,y)\hfill \\ g_2^{\prime }(x,y)=a(x,y)/2\hfill \\ g_3^{\prime }(x,y)=a(x,y)-b(x,y)\cos {\phi }_H(x,y)\hfill \\ g_4^{\prime }(x,y)=a(x,y)-b(x,y)\sin {\phi }_M(x,y)\hfill \\ g_5^{\prime }(x,y)=a(x,y)-b(x,y)\cos {\phi }_M(x,y)\hfill \\ g_6^{\prime }(x,y)=a(x,y)-b(x,y)\sin {\phi }_L(x,y)\hfill \\ g_7^{\prime }(x,y)=a(x,y)-b(x,y)\cos {\phi }_L(x,y)\hfill \end{array}.$$(4)

Refer to Appendix A for the calculation detail of Eq. (4). (a) The pixels of any one, two, or three of the high-frequency patterns in Eq. (1) are saturated, (b) the pixels of any one or both of the mid-frequency patterns in Eq. (1) are saturated, and (c) the pixels of any one or both of the low-frequency patterns in Eq. (1) are saturated. Then we substitute the pixels of the corresponding pattern in Eq. (4). The background image $a$ in Eq. (1) may also produce intensity saturation because of the high reflection of the measured surface. To this end, we reduced the projected background intensity to half; see Appendix A for details. The captured $\pi$-phase-shifted background intensity will also be approximately reduced by half, i.e., $a(x,y)/2$. When the intensity of a pixel in the original background image is saturated, it will be replaced by the pixel in the $\pi$-phase-shifted background image for wrapped phase calculation.

There are eight cases for the high-frequency wrapped phase calculation, as shown below.

Case 1: Three high-frequency fringe patterns are not saturated. $${\psi }_H(x,y)=\arctan \left[\frac{g_1(x,y)-g_2(x,y)}{g_3(x,y)-g_2(x,y)}\right].$$(5)

Case 2: $g_1(x,y)$ is saturated and replaced by $g_1^{\prime }(x,y)$. $${\psi }_H(x,y)=\arctan \left[\frac{g_2(x,y)-g_1^{\prime }(x,y)}{g_3(x,y)-g_2(x,y)}\right].$$(6)

Case 3: $g_2(x,y)$ is saturated and replaced by $g_2^{\prime }(x,y)$. $${\psi }_H(x,y)=\arctan \left[\frac{g_1(x,y)-2g_2^{\prime }(x,y)}{g_3(x,y)-2g_2^{\prime }(x,y)}\right].$$(7)

Case 4: $g_3(x,y)$ is saturated and replaced by $g_3^{\prime }(x,y)$. $${\psi }_H(x,y)=\arctan \left[\frac{g_1(x,y)-g_2(x,y)}{g_2(x,y)-g_3^{\prime }(x,y)}\right].$$(8)

Case 5: $g_1(x,y)$ and $g_2(x,y)$ are saturated and replaced by $g_1^{\prime }(x,y)$ and $g_2^{\prime }(x,y)$. $${\psi }_H(x,y)=\arctan \left[\frac{2g_2^{\prime }(x,y)-g_1^{\prime }(x,y)}{g_3(x,y)-2g_2^{\prime }(x,y)}\right].$$(9)

Case 6: $g_2(x,y)$ and $g_3(x,y)$ are saturated and replaced by $g_2^{\prime }(x,y)$ and $g_3^{\prime }(x,y)$. $${\psi }_H(x,y)=\arctan \left[\frac{g_1(x,y)-2g_2^{\prime }(x,y)}{2g_2^{\prime }(x,y)-g_3^{\prime }(x,y)}\right].$$(10)

Case 7: $g_1(x,y)$ and $g_3(x,y)$ are saturated and replaced by $g_1^{\prime }(x,y)$ and $g_3^{\prime }(x,y)$. $${\psi }_H(x,y)=\arctan \left[\frac{g_2(x,y)-g_1^{\prime }(x,y)}{g_2(x,y)-g_3^{\prime }(x,y)}\right].$$(11)

Case 8: Three high-frequency fringe patterns are saturated. $${\psi }_H(x,y)=\arctan \left[\frac{2g_2^{\prime }(x,y)-g_1^{\prime }(x,y)}{2g_2^{\prime }(x,y)-g_3^{\prime }(x,y)}\right].$$(12)

Equations (5)–(12) can be expressed by the following general formula: $${\psi }_H(x,y)=\arctan \left\{\frac{{(-1)}^{C1}[G_1(x,y)-G_2(x,y)]}{{(-1)}^{C2}[G_3(x,y)-G_2(x,y)]}\right\}.$$(13)

In Eq. (13), if $g_1(x,y)$ is not replaced, $G_1(x,y)=g_1(x,y)$ and $C1=0$; else, $G_1(x,y)=g_1^{\prime }(x,y)$ and $C1=1$. If $g_2(x,y)$ is not replaced, $G_2(x,y)=g_2(x,y)$; else, $G_2(x,y)=2g_2^{\prime }(x,y)$. If $g_3(x,y)$ is not replaced, $G_3(x,y)=g_3(x,y)$ and $C2=0$; else, $G_3(x,y)=g_3^{\prime }(x,y)$ and $C2=1$.

Similar to Eq. (13), the mid-frequency and low-frequency wrapped phases can be expressed as $${\psi }_M(x,y)=\arctan \left\{\frac{{(-1)}^{C3}[G_4(x,y)-G_2(x,y)]}{{(-1)}^{C4}[G_5(x,y)-G_2(x,y)]}\right\}.$$(14)$${\psi }_L(x,y)=\arctan \left\{\frac{{(-1)}^{C5}[G_6(x,y)-G_2(x,y)]}{{(-1)}^{C6}[G_7(x,y)-G_2(x,y)]}\right\}.$$(15)

In Eqs. (14) and (15), if $g_4(x,y)$ and $g_6(x,y)$ are not replaced, $G_4(x,y)=g_4(x,y)$, $G_6(x,y)=g_6(x,y)$, $C3=0$ and $C5=0$; else, $G_4(x,y)=g_4^{\prime }(x,y)$, $G_6(x,y)=g_6^{\prime }(x,y)$, $C3=1$ and $C5=1$. If $g_2(x,y)$ is not replaced, $G_2(x,y)=g_2(x,y)$; else, $G_2(x,y)=2g_2^{\prime }(x,y)$. If $g_5(x,y)$ and $g_7(x,y)$ are not replaced, $G_5(x,y)=g_5(x,y)$, $G_7(x,y)=g_7(x,y)$, $C4=0$ and $C6=0$; else, $G_5(x,y)=g_5^{\prime }(x,y)$, $G_7(x,y)=g_7^{\prime }(x,y)$, $C4=1$ and $C6=1$.

A mouse surface has large local highlights at the camera’s perspective. Figures 1(a) and 1(b) show the original and $\pi$-phase-shifted fringe patterns, $g_1$ and $g_1^{\prime }$. Figure 1(c) shows the cross sections of Figs. 1(a) and 1(b). Some pixels are saturated in Fig. 1(c). Figure 1(d) is a partial enlargement of Fig. 1(c). When the pixels in fringe $g_1$ are saturated, the pixels in fringe $g_1^{\prime }$ are not saturated, so the saturated pixels of $g_1$ can be replaced by the pixels of $g_1^{\prime }$ for phase calculation. However, there may also be a phenomenon in which pixels are saturated in both original and $\pi$-phase-shifted fringe patterns due to local strong reflections. Obviously, the proposed method (PM) cannot correctly calculate the phase for such pixels, but such pixels are relatively small. Therefore, the PM can still greatly reduce the phase error introduced by intensity saturation.

Our FPP system includes a digital light processor (DLP) from Texas Instruments (TI) with a resolution of $912\mathrm{pixels}\times 1140\mathrm{pixels}$, a camera from Daheng Imaging with a resolution of $1024\mathrm{pixels}\times 1280\mathrm{pixels}$, and a computer.

In the first experiment, the measured object is a Peking Opera Mask, and a captured fringe is shown in Fig. 2(a). Figure 2(b) shows 3D reconstruction by the traditional method (TM), and Fig. 2(c) shows 3D reconstruction by the PM. It can be seen that the reconstructed surface based on the TM has a missing surface due to intensity saturation, while the reconstructed surface based on the PM is more complete. To further demonstrate the robustness of the PM, a large-area and strongly reflective object shown in Fig. 3(a) is used for 3D measurement. 3D measurement of such an object is the most challenging. Figure 3(b) shows 3D reconstruction by the TM, and Fig. 3(c) shows 3D reconstruction by the PM. By comparing Figs. 3(b) and 3(c), it can be seen that the error introduced by saturated pixels based on the PM is greatly reduced, which further confirms the effectiveness of the PM.

Experimental comparisons using the TM, the adaptive fringe projection method (AFPM), the multiple exposure adjustment method (MEAM), and the PM are carried out. The measured object is a metal product manufactured by a high-speed computerized numerical control (CNC) machining center. Figure 4(a) shows a fringe $g_4$ with a frequency of 15/912 in a set of fringe sequence using the TM. The PM requires an additional set of $\pi$-phase-shifted fringe sequences. Figure 4(b) shows a fringe $g_4^{\prime }$ in the additional set of $\pi$-phase-shifted fringe sequence. The pixel gray values in fringes $g_4$ and $g_4^{\prime }$ are approximately inversely cosine-distributed. The key to the AFPM is the generation of an adaptive projection fringe sequence. A fringe with a frequency of 15/912 of the adaptive projected fringe sequence generated in this work is shown in Fig. 4(c). The MEAM (24 exposures) is used, and 24 sets of fringe sequences are generated. We extract the optimal results with the largest gray scale but not saturated from the 24 sets of fringes, and perform phase calculation based on the optimal fringe sequence. Figure 4(d) shows a fringe $g_1$ with a frequency of 180/912 when the exposure time is 15 ms. The captured fringe contain intensity-saturated regions, such as the pixel intensity of the ①, ②, and ③ regions in Fig. 4(d). Figure 4(e) shows a fringe $g_1^{\prime }$ in the optimal set of fringe sequences. Obviously, the pixel intensity of the ①, ②, and ③ regions in Fig. 4(e) is suppressed. When the exposure time is wide enough from small to large and the number of exposures is very large, a good final fringe sequence can be extracted by MEAM for phase calculation. Figure 4(f) shows the 3D reconstructed surface using the TM. There are a large number of 3D surface deletions on the highly reflective surfaces, such as the ① and ② surfaces shown in Fig. 4(f). Figure 4(g) shows the 3D reconstructed surface using the AFPM. The 3D surface deletions on the surfaces ① and ② have been reduced. Figure 4(h) shows the 3D reconstructed surface using the PM. Compared with the surface ② in Fig. 4(g), the 3D reconstruction result of the surface ② in Fig. 4(h) is better. However, the 3D reconstruction result of the ① surface in Fig. 4(h) is slightly worse than that of the ① surface in Fig. 4(g). Figure 4(i) shows the 3D reconstructed surface using the MEAM (24 exposures). We can see that the 3D reconstruction shapes on the surfaces ① and ② in Fig. 4(i) is the best of that reconstructed by the four methods.

The comparisons in Fig. 4 confirm that the MEAM (24 exposures) has the best 3D measurement result. However, the large number of fringes, uncertainty in exposure time range, difficult quantification of the number of exposures, and complex operation limit the application of the MEAM. However, to quantify the measurement accuracy of the other three methods, we take the phase extracted by the MEAM (24 exposures) as the standard value and employ a sum of the phase error (SPE) to evaluate measurement accuracy. SPE is defined as $$\mathrm{SPE}={\sum }_{y=1}^w{\sum }_{x=1}^h[{\phi }_{24-\mathrm{exposure}}(x,y)-\phi (x,y)],$$(16)where $w$ and $h$ are the width and height of the captured fringe respectively, and ${\phi }_{24-\text{exposure}}$ is the unwrapped phase extracted by the MEAM.

SPE comparisons of the metal product are shown in Table 1. The SPE of the PM is 81.4% lower than that of the TM, and 61.5% lower than that of the AFPM. To compare the measurement accuracy more comprehensively, we also introduced the number of phase errors (NPEs) at different phase error thresholds to evaluate the measurement accuracy, as shown in Fig. 5. It can be seen from the comparisons in Table 1 and Fig. 5 that the 3D surface reconstructed by the TM is the worst. The 3D surface reconstructed by the PM is better than that using the AFPM.

To further confirm the above results, we performed 3D reconstruction on a wireless mouse with high reflectivity; the results are shown in Fig. 6. Figure 6(a) shows the 3D reconstructed surface using the TM. There are a larger number of 3D surface deletions on surface ①. Figure 6(b) shows the 3D reconstructed surface using the adaptive fringe projection method. The 3D surface deletions on surface ① is greatly reduced. Figure 6(c) shows the 3D reconstructed surface using the PM. The 3D surface deletions on surface ① are less than those in Fig. 6(b). Figure 6(d) shows the 3D reconstructed surface using the 24-exposure adjustment method. The result of 3D reconstruction on surface ① based on the 24-exposure adjustment method is the best; however, the measurement efficiency of the 24-exposure adjustment method is low.

SPE comparisons of a wireless mouse are shown in Table 2. The SPE of the PM is 76.7% lower than that of the TM, and 29.7% lower than that of the AFPM.

Table 3 shows the comparison of the calculation time of each algorithm. Adaptive fringes generation using the AFPM is time-consuming, but the algorithm of the AFPM is the same as that of the TM, so the time cost of the TM and the AFPM is the same, 0.93 s. The time cost of the MEAM is 33.82 s, and the time cost of the PM is 1.95 s. However, the measurement efficiency is also related to the number of fringes and the manual adjustment of the equipment. The manual adjustments of AFPM and MEAM are complicated.

The comparison of the number of fringes is shown in Table 4. For the TM based on the $2f_H+a+2f_M+2f_L$ algorithm, the number of fringes is seven. The number of fringes based on the PM is $2\times 7=14$, the number of fringes based on the AFPM is at least $3\times 7+m>21$ (pixel mapping based on the horizontal and vertical unwrapped phases requires $2\times 7=14$ fringes, adaptive projection requires 7 fringes, and projection intensity estimation requires $m$ fringes, $m=18$ in this work), and the number of fringes based on the MEAM (24 exposures) is $24\times 7=168$. Compared with the number of fringes of the AFPM and MEAM, the PM improves by 64.1% and 91.7%, respectively.

Phase errors caused by the intensity saturation can also be corrected effectively with the standard $N$-step phase-shifting algorithm as long as the phase shift is large enough. To compare the efficiency and accuracy of 3D reconstruction, we take a high-precision standard part processed by the CNC machine as the measured object, and compared the 3D reconstruction efficiency and accuracy of the TM, the PM, and the combination of the eight-step phase-shifting method and three frequency temporary phase unwrapping algorithm (3F-8S), respectively. Table 5 shows the comparison of the number of fringes of the TM, PM, and 3F-8S. Table 6 shows the comparison of the total time cost between TM, PM, and 3F-8S. The calculation time cost of the 3F-8S is 1.82 s. Since the frame rate of the measurement system is 10 Hz, 3F-8S takes $0.1\times 10=1\mathrm{s}$ longer than the PM to capture fringe patterns.

Figures 7(a), 7(b), and 7(c) show a high-frequency, a mid-frequency, and a low-frequency captured fringe pattern, respectively. Surface ① of the fringe pattern contains intensity saturation. Figures 7(d), 7(e), and 7(f) show the extracted unwrapped phases based on the TM, PM, and 3F-8S. For local comparison, we extract a row of cross sections of the unwrapped phases; the results are shown in Fig. 7(g). It can be seen from the unwrapped phase comparison that the unwrapped phases extracted based on the PM and 3F-8S are similar, and much better than that of the TM.

Figure 8(a) shows a high-precision standard part processed by the CNC, whose computer-aided design (CAD) model has an ideal thickness of 8 mm. After CNC machining, we actually measured the thickness to be 8.01 mm, as shown in Figs. 8(b) and 8(c).

We take the actual measured height (thickness) of 8.01 mm as the standard height, and use the TM, PM, and 3F-8S to obtain the 3D point clouds, as shown in Fig. 9. The height errors are shown in Table 7. On the non-intensity saturated surface, the height errors of the three positions using TM are 0.036, 0.058, and 0.069 mm, respectively, where the maximum height error is 0.069 mm and the average height error is 0.054 mm. The height errors of the three same positions using the PM are 0.035, 0.006, and 0.06 mm, respectively, where the maximum height error is 0.06 mm and the average height error is 0.034 mm. The height errors of the three same positions using 3F-8S are 0.03, 0.021, and 0.055 mm, respectively, where the maximum height error is 0.055 mm and the average height error is 0.035 mm.

However, only comparing the height error of the unsaturated surface is incomplete because the above three methods can well reconstruct the 3D shape for the unsaturated surface. We can see that the 3D point cloud using the TM has holes in the intensity-saturated surface, while the 3D point clouds using PM and 3F-8S are complete in the intensity-saturated surface. For comprehensive comparison, we take the 3D point cloud using the 24-exposure method as the standard value, and use the root mean squared error (RMSE) for comparison; the results are shown in Table 7. The RMSE using PM is 86.7% lower than that using the TM, and the RMSE using 3F-8S is 87.7% lower than that using the TM. By comparison, it can be seen that the RMSE using the PM is similar to that using 3F-8S, but the number of fringe patterns is reduced by 41.7%.

(i)For the MEAM, satisfactory 3D measurement results can be obtained as long as the exposure range and number are large enough. However, for different objects with high reflective properties, the exposure range is difficult to predict, the exposure time cannot be quantified, and the large number of fringes leads to a great decrease in measurement efficiency. Therefore, the application of the MEAM is limited.(ii)For the AFPM, it needs to precapture images according to the camera and estimate the projection intensity, but the estimation of projection intensity is limited in two aspects. One is the pixel mapping error between the camera and projector. Since the reflection intensity of the measured surface needs to be estimated in the camera coordinate and transformed into the projector coordinate, it is necessary to establish a pixel mapping between the camera and projector. The other is the dynamic range of the camera and the projector. For example, the gray-scale range of the camera and the projector is 0–255 for the 8-bit image, but due to the high reflection of the surface, the pixel intensity exceeds this range, resulting in inaccurate projection intensity estimation. Therefore, the AFPM inevitably has certain limitations, and the number of fringes is also much higher than that of the PM.(iii)The PM only needs to additionally project a set of $\pi$-phase-shifted fringe sequences, does not require complex hardware adjustment, and its algorithm is simple. 3D reconstruction error of the PM is also much smaller than that of the AFPM.

For 3D measurement of a non-diffuse surface based on the $2f_H+a+2f_M+2f_L$ algorithm, we propose a method (PM) that combines the original fringes and the $\pi$-phase-shifted fringes. Saturated pixels are replaced by unsaturated pixels in the $\pi$-phase-shifted $2f_H+a/2+2f_M+2f_L$ fringes. We give eight replacement cases and corresponding phase calculation methods in detail. Experiments verify the effectiveness and robustness of the PM. Since the PM does not require numerous and difficult-to-quantify exposure adjustment, nor does it require pre-estimation adaptive projection intensity and pixel mapping, it is simple, efficient, and easy to implement.