Angular momentum is one of the most vital and fundamental properties of physical particles.1^–3 Since Allen discovered that vortex beams with a helical wavefront phase distribution $\exp (i\ell \theta )$ carry orbital angular momentum (OAM) in 1992,2 where $\theta$ is the azimuthal coordinate and $\ell$ is the topological charge number, researchers have conducted thorough studies on vortex beams for years, demonstrating its applications in optical manipulation,4^–6 micromachining,7 superresolution imaging,8 and quantum systems.9^,10 The unlimited range of $\ell$ implies that the OAM beam possesses an infinite number of eigenstates that can be exploited to infinitely elevate the capacity of a communication system.11^–13

Due to its significant importance, a variety of techniques have been proposed for the generation of vortex beams, ranging from diffractive optical elements14^–16 and spatial light modulators17 to metasurfaces.18^–21 Additionally, in the OAM-based communication systems such as OAM shift keying22 and OAM division multiplexing,11 a mode sorter capable of effectively sorting different OAM modes is particularly crucial. Therefore, there has been enormous enthusiasm for sorting OAM. Some accomplished schemes contain interference, diffraction, and mode conversion methods.23^,24 Sorting techniques based on Mach–Zehnder interferometers can separate even and odd OAM modes into two ports.25^,26 One optical coordinate transformation method that transforms the azimuthal phase to the horizontal phase provides a simple and efficient approach to sorting vortex beams.27^,28 In addition, metasurfaces designed for the sorting of optical OAM modes have also been developed.29^,30 In addition, diffractive optical neural networks (DONNs) and the multiplane light conversion (MPLC) method can also be used to sort OAM modes.31^,32 All those methods have propelled the development of sorting techniques for optical vortex modes.

However, in the aforementioned methods, effective sorting can only be applied to input modes with integer OAM (IOAM). Actually, $\ell$ can also be a noninteger, corresponding to the fractional OAM (FOAM) beam.33^–37 In 2004, Berry provided the first theoretical analysis of the FOAM mode, proving that it could be regarded as a superposition of IOAM eigenmodes.33 Based on its novel properties (markedly different from IOAM mode),38^,39 the FOAM mode has further expanded the scope of OAM applications, achieving unique functionalities in multiple domains, including the precise control over cell orientation in optical micromanipulation40^,41 and the anisotropic edge enhancement in optical imaging.42^–44 Another promising application is the optical communication system based on FOAM modes, which holds the promise of further expanding communication capacity.45^,46 But the lack of an effective FOAM mode sorter hinders its further development. One method utilizing machine-learning algorithms enables the high-resolution identification rather than the effective sorting of FOAM modes.47 Therefore, it is of fundamental importance to develop an effective sorting technique applicable to FOAM input modes.

In this paper, we propose a method for the effective sorting of fractional vortex modes. Following the coordinate transformation method, we first convert the FOAM noneigenmodes into IOAM eigenmodes. Subsequently, the transformed IOAM beam is effectively sorted using a mode conversion method called topological charge matching, which is commonly used in OAM-based optical communication.11 To evaluate its performance, we conduct sorting experiments on different groups of FOAM modes and perform the purity analysis on the sorted FOAM modes. The experimental $>86\%$ purity verifies the success of our new approach in effective sorting of fractional vortex modes.

The coordinate transformation theory in ray optics enables specific mapping operations on the spatial position of light. Considering the paraxial propagation of a parallel light between two planes, where there is a distance $d$ between the input plane $(x,y)$ and the output plane $(u,v)$, the generalized Snell’s law says that there exists a coordinate transformation between points on two planes, which is described by48$${\mathcal{Q}}_x=k\frac{u-x}{d},{\mathcal{Q}}_y=k\frac{v-y}{d},$$(1)where $k$ is the wave vector in free space and $\mathcal{Q}(x,y)$ is the transformation phase loaded on the input plane. By loading different $\mathcal{Q}(x,y)$, various types of coordinate transformation can be achieved. Spiral transformation is a special transformation from one spiral to another. The input and output planes are redefined in terms of a spiral-polar coordinate described by the radial coordinate $r$ and the spiral azimuthal coordinate varies in $[0,+\infty )$. The entire plane can be partitioned by a set of spirals with no $2\pi$ restriction on angular coordinates, which allows for finer divisions of the plane.

The specific transformation laws for spiral transformation are described by $$\rho =c{r}^{-1/n},\phi =\theta /n,$$(2)where $(r,\theta )$ and $(\rho ,\phi )$ are the coordinates of the input and output planes, $c$ is an arbitrary constant, and $n$ is the transformation factor. Assuming that $(r,\theta )$ lies on a spiral defined by $r=a{e}^{b\theta }$ which is used in our method, according to Eq. (2), $(\rho ,\phi )$ will also lie on a new spiral $\rho =c{a}^{-1/n}{e}^{-b\phi }$. Based on Eqs. (1) and (2), the required transformation phase ${\mathcal{Q}}_1(r,\theta )$ on the input plane is calculated as (see Sec. 1 in the Supplementary Material for details) $${\mathcal{Q}}_1(r,\theta )=\frac{k}{d}[\frac{c{r}^{1-1/n}}{1-1/n}\cos (\theta -\theta /n)-\frac{r^2}{2}].$$(3)

Another phase mask ${\mathcal{Q}}_2(\rho ,\phi )$ (correction phase) on the output plane is required to compensate for both ${\mathcal{Q}}_1(r,\theta )$ and the propagation phase generated during the propagation in free space, which is written as $${\mathcal{Q}}_2(\rho ,\phi )=-{\mathcal{Q}}_1-k\sqrt{r^2+{\rho }^2-2r\rho \cos (\phi -\theta )+d^2}.$$(4)

For FOAM input modes, an additional correction phase $\mathcal{P}$ is superimposed onto ${\mathcal{Q}}_2$, which is expressed as $$P(\rho ,\phi )=2\pi t\left[\frac{n\phi }{2\pi }\right],$$(5)where $t=\mathrm{mod}({\ell }_{\text{in}},1)$. Such a requirement stems from the unique characteristic of the FOAM modes—its phase distribution in the azimuthal direction is discontinuous. The spiral transformation compresses or stretches the input optical field by the factor of $n$ along the azimuthal direction without directly changing the phase. The discontinuities in the azimuthal phase persist after spiral transformation, leading to the azimuthal phase jump on the output beam, for which the correction phase $\mathcal{P}$ is introduced to compensate (see Sec. 2 in the Supplementary Material for details).

If the incident beam is a Gaussian vortex beam with the waist radius $w_0$ and a topological charge ${\ell }_{\text{in}}$, the amplitude of the field $E_1$ before the transformation is given by $$E_1(r,\theta )=\frac{1}{w_0}{\left(\frac{r\sqrt{2}}{w_0}\right)}^{|{\ell }_{\text{in}}|}\exp (-\frac{r^2}{w_0^2})\exp (i{\ell }_{\text{in}}\theta ).$$(6)

After the transformation and the phase correction, the field $E_2$ at the output point $(\rho ,\phi )$ is equal to the field $E_1$ at the corresponding input point $(r,\theta )$ multiplied by a factor, which is given by $$E_2(\rho ,\phi )=-i\exp (ikd)|n|\frac{r}{\rho }{E}_1(r,\theta ).$$(7)

The relationship between $(r,\theta )$ and $(\rho ,\phi )$ is given by Eq. (2). Therefore, the exponential term in Eq. (7) is $\exp (i{\ell }_{\text{in}}\theta )=\exp (\mathit{\text{in}}{\ell }_{\text{in}}\phi )$, which indicates that the transformed beam is a vortex beam carrying OAM with ${\ell }_{\text{out}}=n{\ell }_{\text{in}}$.

Figure 1 presents the theoretical prediction of the phase distribution of the optical field before and after our spiral transformation with different FOAM input modes ${\ell }_{\text{in}}$ and transformation factors $n$. The second and the third columns of Figs. 1(a)–1(c) illustrate the phase distribution of the output beam after the transformation without and with $\mathcal{P}$, respectively. The phase distributions of the transformed beam without $\mathcal{P}$ possess azimuthal jumps in the horizontal direction, which are entirely different from the intended integer vortex modes. The results with $\mathcal{P}$ demonstrate the successful conversion from FOAM modes to target IOAM modes. The successful mode conversion requires that the input vortex field is completely split by the spiral and that the output field can form a complete vortex shape, which means that we must select appropriate parameters $a$ and $b$ of the spiral (see Sec. 3 in the Supplementary Material for details).

The output vortex beam obtained after spiral transformation will be sorted by topological charge matching. The concrete operation is applying a vortex phase map with a topological charge of ${\ell }_m$ to the output beam, after which the amplitude becomes $$\begin{gathered} E_3(\rho ,\phi )=E_2(\rho ,\phi )\times \exp (i{\ell }_m\phi ) \\ \propto \exp [i({\ell }_{\text{out}}+{\ell }_m)\phi ]. \end{gathered}$$(8)

Obviously, when the topological charges match each other $({\ell }_{\text{out}}+{\ell }_m={\ell }_{\text{all}}=0)$, the exponential term representing OAM will be completely eliminated, which means that after focusing through a lens, the intensity distribution on the focal plane will revert back from a halo to a Gaussian-like spot in the center of the field. In cases of ${\ell }_{\text{all}}\ne 0$, the halo still exists. Therefore, we can sort the target vortex mode of ${\ell }_{\text{in}}$ using a spatial filter (SF) to extract the central spot after adding the vortex phase map of ${\ell }_m=-{\ell }_{\text{out}}=-n{\ell }_{\text{in}}$. Since the FOAM modes are not the eigenstates, the direct use of the topological charge-matching method will cause strong cross talk.49 After our coordinate transformation, the FOAM modes are transformed into IOAM eigenmodes, thereby enabling effective sorting of such FOAM states with low cross talk.

The numerical simulations of the effective sorting of FOAM modes are performed by rigorously computing the Fresnel integral on the incident light field, which are shown in Fig. 2. The parameters involved in the simulations are $d=1\mathrm{cm}$, $k=2\pi /\lambda$, $\lambda =532\mathrm{nm}$, $r_0=240\mu \mathrm{m}$, $w_0=100\mu \mathrm{m}$, $b=0.05$, and $c=r_0^{1+1/n}$. Figures 2(a)–2(d) show the numerical results of the intensity distribution, phase distribution, and topological charge matching of the transformed beam on the output plane. From top to bottom, the input topological charges are ${\ell }_{\text{in}}=3/2,5/2,4/3,7/3$, and the transformation scaling factors are $n=2,2,3,3$, respectively. The corresponding target IOAM modes are ${\ell }_{\text{out}}=3,5,4,7$. The first two columns are the intensity distribution and the phase distribution of the transformed beam, while the last five columns are the results of topological charge matching. To better demonstrate the functionality of topological charge matching in sorting vortex modes, we simulated the results under conditions of both matching $({\ell }_{\text{all}}=0)$ and mismatching $({\ell }_{\text{all}}=\pm n,\pm 2n)$. There is no need to use other ${\ell }_m$, since there are no matching IOAM modes. The condition of ${\ell }_{\text{all}}=0$ means the topological charge of the added vortex phase map is the opposite of the target IOAM mode (${\ell }_m=-n{\ell }_{\text{in}}$). Therefore, we can determine the success of the sorting based on the presence of a single Gaussian spot appearing in the center of the light field under the condition of ${\ell }_{\text{all}}=0$.

The corresponding numerical results align well with our theoretical predictions. As shown in Fig. 2, the intensity distributions exhibit a spiral vortex pattern. There are several intensity breakpoints on the spiral, which originate from the radial opening ring pattern of the FOAM modes. One can clearly see that the phase distribution exhibits a good continuity with no azimuthal phase jumps, demonstrating continuous cyclic variations from 0 to $2\pi$ that are repeated $n{\ell }_{\mathrm{in}}$ times, which is exactly consistent with the target IOAM mode. The successful transformation from FOAM to IOAM mode is further validated by the results of topological charge matching. It is evident that for all the FOAM input modes, the intensity distribution of the output field is a single Gaussian-like spot in the center of the field when ${\ell }_{\text{all}}=0$. The result for ${\ell }_{\text{all}}=\pm n,\pm 2n$ is a light ring significantly larger than the spot. There is almost no overlapping region between the spot and the light ring, indicating that the cross talk between different FOAM modes is very low when using SF for sorting. We also quantitatively characterize the cross talk between different OAM modes by calculating the overlap integral between the optical fields (see Sec. 4 in the Supplementary Material for details). Such simulation results confirm the validity of our scheme to effectively sort the FOAM modes.

Furthermore, we carry out an experiment to verify the theoretical results above. The phase modulation of the FOAM mode is achieved using a spatial light modulator (SLM). The phase maps loaded onto the SLM are calculated with the formulas mentioned above. Specifically, the transformation phase map is obtained by ${\mathcal{Q}}_1$, and the correction phase map is calculated using ${\mathcal{Q}}_2+\mathcal{P}$. The parameters used in the calculations remain consistent with those employed in simulations. The results are converted into gray-scale images suitable for SLM. Figures 3(a) and 3(b) present the phase maps that are applied to experiment, in which the transformation phase and the correction phase are presented from left to right.

The experimental setup is illustrated in Fig. 3(c). The light source is a continuous wave laser with the wavelength of 532 nm. The polarization of the laser beam is changed to the horizontal direction by using the polarizer (P). Subsequently, the laser beam is reflected by SLM1, which is loaded with a vortex phase map to produce input FOAM states. The SLM we use is a reflective phase-only liquid crystal modulator (UPOLabs, HDSLM80R), which has a rate of 60 Hz and a resolution of $1920\text{pixel}\times 1080\text{pixel}$. The input light is illuminated onto SLM2 loaded with a transformation phase map via a $4f$ system composed of lenses L1 and L2. The focal lengths of the lenses used in the experiment are all $f=100\mathrm{mm}$. The modulated beam is directed onto SLM3 loaded with the correction phase through another $4f$ system consisting of L3 and L4, where SLM3 is located on the back focal plane of this $4f$ system. There is a distance of $f+d$ between SLM2 and L3, which ensures the free propagation of light between the input and output plane at a distance of $d$, as required by the coordinate transformation principle. A low-pass spectral filtering process is implemented via a third $4f$ system with an SF in the intermediate Fourier plane to isolate the desired IOAM beam. A spiral phase plate (SPP) with topological charge ${\ell }_m$ is placed at the back focal plane of L6 to implement topological charge matching. Finally, the output beam is focused by L7 and imaged by a CMOS camera (CCD) on the focal plane. The SPP, L7, and CCD collectively form the OAM detection system. As previously mentioned, the effectiveness of our scheme will be verified based on the results of topological charge matching captured by the CCD. In addition, during the detection of the intensity distribution, the SPP and L7 are removed, and the CCD is relocated on the back focal plane of L6 to directly capture the image of the output beam.

Figure 4 summarizes the experimental results of effective sorting of fractional vortex modes. Our sorting method is effective for any set of FOAM modes with the same $t$. To verify this, we conduct sorting experiments on two sets of fractional vortex modes. Figure 4(a) shows the results for FOAM modes with $t=1/2$ and $n=2$, including ${\ell }_{\text{in}}=3/2,5/2,7/2,9/2,11/2$. Figure 4(b) shows the results for FOAM modes with $t=1/3$ and $n=3,$ including ${\ell }_{\text{in}}=4/\mathrm{3,7}/3,10/3,13/3,16/3$. The corresponding transformation phase and correction phase are illustrated in Figs. 3(a) and 3(b). The first two columns in each group are the intensity distribution before and after the transformation. As expected, the spiral vortex distribution is observed in all groups. The results of sorting by topological charge matching also match well with the numerical simulations. The ${\ell }_m$ of the vortex maps used are equal to $-n{\ell }_{\text{in}}$; i.e., from $-3$ to $-11$ in Fig. 4(a) and from $-4$ to $-16$ in Fig. 4(b). It can be clearly seen that for each FOAM input mode, only when the ${\ell }_m$ and the target topological charges ${\ell }_{\text{out}}$ match each other $({\ell }_{\text{all}}=0)$, the intensity distribution obtained on CCD is a single Gaussian-like spot in the center of the field. Under the condition of mismatching, the intensity pattern is a halo larger than the Gaussian-like spot. The agreement of the experimental results with the simulations is quite satisfactory in terms of the shape and the position of the results of topological charge matching, confirming the successful conversion from FOAM modes to IOAM modes. In each column, there is almost no overlap between the Gaussian-like spot and the halos, indicating that our scheme allows for effective sorting with low cross talk of such a set of FOAM modes by using the SF to extract the central Gaussian-like spot. It is also noted that the patterns are surrounded by some stray light in the background, which is attributed to the imperfect phase matching in the process caused by small errors in the experimental setup, such as transverse displacement and the distance between SLMs.

Further results demonstrating the ability of our scheme for effective sorting of FOAM modes is the OAM mode purity analysis of the extracted beam. Following the principle of topological charge matching, in the case of using the same matched vortex phase map of ${\ell }_m$, we can determine the mode purity by measuring the intensity in the region of the central Gaussian-like spot for all the input states (see Sec. 5 in the Supplementary Material for details). The purity of different input states in the extracted beam is the ratio of each measured intensity to the total measured intensity. Figure 5 shows the numerical and experimental results for both sets of FOAM modes. Since there is little overlap between the patterns of two neighboring FOAM states, the results indicate high purity of the target FOAM mode in the extracted beam, which verifies the success of effective sorting with low cross talk of fractional vortex modes.

In the present form, our method has demonstrated its powerful ability to effectively sort arbitrary FOAM modes, yet it can only extract one FOAM mode at a time by the topological charge matching method. We would like to point out that it can be improved by combining other efficient IOAM sorting methods,27^,28 indicating the potential of our method to further realize the efficient sorting of FOAM modes. It is worth mentioning that the communication system based on FOAM modes has significant advantages in terms of confidentiality, as they are more difficult to identify and sort compared to IOAM modes. In this regard, our method can serve as an encryption key for implementing highly confidential FOAM-based communication systems. And, since the coordinate transformation method has no wavelength restriction, our method can also be generalized to other wave bands.

In addition to coordinate transformation methods, other schemes hold promise for the effective classification of FOAM modes. The traditional MPLC method31 utilizes an important transformation between Cartesian points and the Cartesian indices of Hermite-Gaussian modes, which is only applicable to IOAM modes. The MPLC method combined with DONN32 has more powerful features. The feasibility of machine learning methods for classifying FOAM modes has been experimentally demonstrated.47 Therefore, the DONN method may be able to classify FOAM modes after suitable training, but the generalizability of this approach is limited by the training set and its actual performance needs further validation in experiments.

We have theoretically and experimentally reported a novel scheme based on coordinate transformation to implement the effective sorting of fractional vortex modes, for the first time to our best knowledge. Starting from the theoretical predictions, we propose a coordinate mapping method to convert FOAM noneigenmodes to IOAM eigenmodes, after which the effective sorting of different FOAM modes can be implemented by topological charge matching. The theoretical predictions have been verified through numerical simulations and experiments. We use FOAM modes with 2 and 3 in the denominator to perform the proof-of-principle experiments. Furthermore, the OAM mode purity analysis confirms that the sorting of FOAM modes has low cross talk. With the ability to perform effective sorting of fractional vortex modes, our work paves the way for the FOAM-based communication and data storage system. More attractively, our scheme can further provide encryption for the communication process, which will open up new applications in both the classical and the quantum regimes.

Biographies of the authors are not available.