Since the discovery by Allen et al.1 in 1992 that beams with helical phase distributions carry orbital angular momentum (OAM) modes, optical vortices (OVs) have attracted significant attention and extensive research due to their unique physical properties. Over time, a variety of OV modes have been identified, including OAM modes,1^–3 cylindrical vector beams (CVBs),4 and generalized vortex beams5 in free space, plasmon vortices6^,7 on metal surfaces, and spatiotemporal vortices8^,9 discovered in the spatiotemporal domain. Among them, OVs such as scalar vortex beams or vector vortex beams carrying OAM or CVB modes have achieved notable success in the field of optical communications, owing to their infinite orthogonality and passive controllability.10^–14 These modes offer unprecedented dimensions for information encoding, providing the way for innovations in optical communications, particularly in classical optical communications, enabling ultra-high information density and enhanced communication capacity.

Mode (de)modulation, mode processing, and mode transmission are the three fundamental pillars for the construction of mode-division communication systems and their practical implementation in networking. Mode (de)modulation, which involves encoding or decoding signals on OAM or CVB modes,12^,15^–17 constitutes the foundational step in utilizing these modes for information. Mode processing technologies, encompassing mode multiplexing and demultiplexing,13^,14 mode add/drop,18^,19 mode routing,20 and mode computing,21 are critical for manipulating the mode field distribution of OVs to couple, separate, and facilitate interactions among multiple OAM/CVB modes. These processes are essential for achieving interconnectivity between optical network nodes, as well as between data centers and optical terminals. Furthermore, given that OVs with spiral phase wavefronts are prone to distortion during propagation, mode transmission technologies are indispensable for overcoming atmospheric turbulence aberrations in free space,22^,23 confining OVs within waveguides,24^,25 and enabling stable long-distance transmission. These mode transmission technologies are crucial for ensuring the reliable and efficient propagation of OVs across various communication scenarios.

In this review, we focus on the historical development and recent advancements in OAM/CVB mode-based optical communications, examining the topic from three key perspectives: mode modulation, mode processing, and mode transmission. We provide a comprehensive overview of the fundamental principles and techniques involved in both single-mode and multi-dimensional mode modulation, with particular emphasis on the complexities of high-dimensional modulation techniques and the demodulation strategies that enable fast and accurate mode recognition. In addition, we explore mode processing techniques, including mode demultiplexing, interconnection, routing, and computing, which are essential for establishing the core functionalities required to construct mode-divided communication networks. Furthermore, we delve into the development of mode transmission in both free-space optics and fiber waveguides, addressing challenges posed by atmospheric disturbances and the requirements for long-distance transmission. We discuss various approaches and technologies designed to mitigate these disturbances, ensuring stable and reliable mode transmission. This comprehensive review aims to provide a detailed understanding of the progress made in optical communication systems based on OVs, particularly in terms of channel capacity, bit error rate (BER), and other critical metrics. Ultimately, it contributes to the ongoing development and practical implementation of optical communication networks utilizing optical vortex modes.

The structure of the subsequent sections is arranged as follows. In Sec. 2, we provide a brief overview of OAM/CVB mode-based communication systems and networks, highlighting key perspectives and related technologies in mode modulation, mode processing, and mode transmission. In Sec. 3, we review works on mode modulation and demodulation, emphasizing the complexities and strategies associated with higher-order mode encoding and fast mode recognition. In Sec. 4, we explore the fundamental mode processing techniques essential for constructing mode-divided communication networks, including mode (de)multiplexing, interconnection, and computing. In Sec. 5, we outline the advancements and challenges in mode transmission within free space and optical fiber. Finally, we discuss the current challenges faced by OAM-based communication systems and their networking from various perspectives, identifying potential future research directions.

Linear and angular momenta are fundamental physical properties of light beams. Linear momentum is determined by the amplitude and propagation direction of photons, whereas angular momentum is characterized by the polarization and spatial wavefronts of light. Linear momentum has been utilized in optical communications by encoding information into the amplitude of light and employing higher-order modulation formats. Angular momentum, which includes spin angular momentum (SAM) and orbital angular momentum (OAM), provides additional degrees of freedom for information multiplexing, thereby enhancing communication capacity. SAM is an intrinsic form of angular momentum associated with the transverse polarization oscillation of light, with its sign corresponding to left-handed and right-handed polarization states. OAM, on the other hand, arises from the helical or spiral wavefront of a light beam, in which the wavefront twists such as a corkscrew around the propagation axis, representing the orbital rotation of the light beam. In free space, various types of light beams carry OAM, including Laguerre-Gaussian (LG) beams, which are among the most widely used beams for transmitting OAM modes and can be described in cylindrical coordinates:26$$\begin{gathered} uLG(r,ϕ,z)=2p!π(p+|l|)!1w(z)[r2w(z)]|l|exp[−r2w2(z)] \\ ⁢Lp|l|[2r2w2(z)]exp[ikr22q(z)]exp[−iφpl(z)]exp(ilϕ), \end{gathered}$$(1)where (r,ϕ) represent the radial distance and azimuthal angle in cylindrical coordinates, p and l are the radial and angular indices that describe the spatial distributions. w(z)=w0[(z2+zR2)/zR2]1/2 represents the waist radius of the beam at the transmission distance of z, w0 is the initial waist radius, and zR is the Rayleigh distance. k is the wave vector, and q(z)=(z2+zR2)/z. φpl(z)=(2p+|l|+1)arctan⁡(z/zR) is the Gouy phase shift, and Lp|l| is the generalized Laguerre polynomial. According to this equation, the optical characteristics of the LG beam are mainly determined by p and l, where p controls the number of rings in the light beam, and l controls the topological charge (TC).

The beams carrying OAM modes mentioned above possess helical wavefronts but maintain uniform polarization distributions. By contrast, another type of structured light beam featuring polarization singularities is the CVB, which serves as a coaxial approximation solution to the vector Helmholtz fluctuation equation. The generation of CVB modes is typically achieved through the combination of two orthogonal basis components: left-handed polarization and right-handed polarization. The mathematical description of CVBs is provided below:14$$\begin{gathered} Evector=E0[cos(mϕ+φ0)sin(mϕ+φ0)] \\ =12E0exp(−iφ0)exp(−imϕ)[1i]+12E0exp(iφ0)exp(imϕ)[1−i]. \end{gathered}$$(2)

According to this equation, a polarization order m can be decomposed into a linear superposition of a left-circularly polarized vortex beam with topological charge m and a right-circularly polarized vortex beam with topological charge −m, with both components having equal amplitudes. These two structured modes in OVs provide a wealth of degrees of freedom for optical communication, significantly enhancing communication capacity and spectral efficiency. Figure 1 illustrates the higher-order Poincaré sphere, which represents both the helical phase and polarization distributions of OVs. The left-handed and right-handed polarized beams with opposite topological charges are located at the poles, whereas other points on the sphere correspond to superpositions of these polarization states. Notably, CVB modes with radial and azimuthal polarizations are positioned along the equator, resulting from the composition of OAM modes with orthogonal polarizations and opposite topological charges. The intensity and phase distributions of OAM modes vary with their topological charges. As the topological charge increases, the waist radius of the beams also expands, and the angular period of the phase correlates with the topological charge. The CVBs can be expressed as combinations of two circularly polarized vortex beams with opposite spins and topological charges.

Figure 2 illustrates the schematic diagram of a mode-division optical communication system, which resembles a conventional optical communication system. There are three main technologies involved in mode-division communication networks: mode (de)modulation, mode processing, and mode transmission. The mode modulation technique entails modulating a digital signal onto a vortex beam by converting it into a coaxially transmitted mode signal. This approach enables high-order modulation by switching modes to encode the digital signal, taking advantage of the infinite quadrature property of modes. Demodulation involves analyzing the mode components of the multiplexed channel and rapidly identifying them. Mode channel processing technologies serve as an “information hub,” providing tools for manipulating light fields and reallocating their propagation directions. Researchers have primarily concentrated on mode demultiplexing technologies, which transform multiple off-axis Gaussian beams with encoded data flows into coaxially propagating on-axis light beams. This strategy can significantly enhance communication capacity by leveraging the orthogonality of modes.

However, merely increasing capacity without effective channel processing cannot advance the practical networking applications of optical communication networks. Essential operations such as uploading, downloading, routing, mode filtering, and computation require technologies that manipulate mode channels at various optical nodes and terminals to enhance flexibility. The transmission of optical vortex modes presents significant challenges for practical applications, primarily due to (1) propagation dispersion of vortex beams caused by their helical wavefronts and (2) atmospheric turbulence that distorts phase wavefronts, ultimately degrading communication quality. For long-distance fiber transmission, specialized equipment can efficiently couple multiple modes while minimizing attenuation and crosstalk losses. In free-space propagation, an adaptive compensation system equipped with multiple focal lenses can help mitigate the effects of dispersion (Fig. 3).

Data streams are commonly modulated onto the amplitude distributions of light beams, with modulation capabilities dependent on the encoding format applied to the amplitude during the modulation process. In this context, higher modulation orders enable more data to be encoded and transmitted per time sequence. However, the demodulation procedures require retrieving signals from the distorted amplitude distribution of light, where accuracy sharply deteriorates as modulation orders increase due to amplitude fluctuations caused by interference during propagation. Recently, OVs that possess phase or polarization singularities have introduced additional physical dimensions for signal modulation in optical communication. The OAM and CVB modes of vortex beams have garnered significant attention for (de)modulation in optical communication due to their mutually orthogonal and unbounded characteristics. The mode modulation scheme typically involves encoding each data symbol with specific modes, representing the data streams as a sequence of vortex beams with different modes. Demodulation is achieved by extracting optical feature information from phase or intensity distributions, thereby establishing mapping relationships for mode identification. In this section, we first introduce mode modulation technologies that encompass mode state modulation and multi-dimensional modulations. We then highlight mode demodulation technologies designed for fast and accurate mode identification, facilitating practical applications in mode shift-keying communications.

Figure 4 illustrates several main mode modulation principles and approaches. Due to the discrete nature of OAM mode, by establishing a mapping relationship between the topological charge of OAM mode and binary data stream, multi-level encoding modulation can be achieved. As shown in Fig. 4(a), by constructing an 8-bit binary sequence and corresponding it to 8 OAM modes with TC=−10,−7,−4,−1,1,4,7,10, it is possible to encode 256 orders into the superposition state of OAM modes and then achieve data coding of OAM mode shift keying by sequentially switching a series of specially designed holograms.27Figure 4(b) shows some examples of the superposition state of different OAM modes. It follows that the maximum error-free coding orders approach 2N, in which N is the number of orthogonal modes within this system. As shown in Fig. 4(c), based on such an encoding method, a gray-scale image was experimentally transmitted in free space for 10 m, and all transmitted mode states were accurately received. The system achieved a zero BER. In addition to using integer-order OAM mode to achieve binary sequence encoding, it can also be achieved through fractional-order OAM mode. Using an 8-bit fractional order OAM pattern with an interval of 0.01 and its correspondence with an 8-bit binary sequence, it is possible to generate a phase distribution related to the binary sequence, thereby encoding the binary sequence into the fractional order OAM pattern.17 Although there is a 2N increasing relationship between the achievable modulation order and the number of modes used, there is a clear upper limit on the modulation order that can be achieved using single-mode modulation due to the limitations of fast divergence and limited receiving aperture.28 There is growing interest among researchers in further increasing the modulation orders used in mode shift-keying communication links. One promising approach being explored is to leverage additional physical dimensions beyond OAM modes to further increase the modulation capabilities, such as spatial positions16^,29 and radial indices.30^–32 In 2017, by optimizing the use of OAM mode and spatial position, Li et al.16 proposed a signal encoding and decoding scheme using an OAM mode array and achieved 625- and 1296-order OAM mode modulation using four spatial positions and five and six OAM modes, successfully transmitting grayscale image signals with BERs below 10−3. By combining OAM mode and radial index, Fu et al.32 proposed a high-dimensional encoding/decoding scheme for multi-ring OVs. It achieves signal encoding by constructing a direct mapping between binary sequences and radial-azimuthal indices, which can further improve encoding efficiency in finite OAM mode states. In addition, due to the independence between the OAM mode and the radial index in the optical vortex, this allows us to stack multiple OAM modes under a single radial index to further increase the modulation order. Using this scheme, Luan et al.30 successfully achieved 768-order LG mode keying communication and transmission of three-channel color images and realized a high image recognition accuracy. Although further improvement in modulation order is achieved by expanding spatial dimensions and using radial exponents, it still faces trade-offs introduced by high-order mode divergence. To address this problem, a phase difference modulation strategy was introduced by Chen et al.33 which enables the continuous variation and generation of a series of spatially encoded modes within the same superposed OAM modal basis. Results show that the constructed OAM-SK communication link achieved a modulation order up to 4×104 with only three OAM (+1, +2, and +3) modes superimposed and the decoding accuracy reached 99.9%, providing new insight for high-order OAM-based SK communication. For the joint modulation of polarization and OAM modes, Liu et al.34 proposed all-dielectric metasurface devices for generating a generalized-perfect Poincaré beam and encoding data at cylindrical and circular Poincaré beams for simultaneous modulations. This enables the transmission of encrypted information and increases the modulation orders, realizing the maximum 50-ary vector mode encoding. Besides the conventional OVs with longitudinal OAM mentioned above, optical spatiotemporal vortices carrying transverse OAM have also been shown in recent studies to provide high-dimensional information encoding dimensions, such as Hopfions35 and spatiotemporal vortex strings.36

On the other hand, efficient and accurate demodulation of modes is essential for shift-keying communication. The primary challenge lies in identifying the OAM modes necessary to recover the transmitted data streams. Existing methods for OAM mode identification can generally be categorized into two types: optical measurement-based techniques and deep learning-assisted approaches. Optical measurement techniques utilize optical interference, diffraction, and the rotational Doppler effect to identify modes. By contrast, deep learning-assisted methods employ feedforward neural networks (FNNs) and convolutional neural networks (CNNs) to extract information and establish mapping relationships based on intensity and optical parameters. In this section, we first discuss the existing optical measurement techniques for OAM mode identification and their characteristics. We summarize the working principles and performance of common methods, such as interference, diffraction, and Doppler measurements. Next, we explore deep learning-assisted approaches, detailing how neural networks are applied to OAM mode classification and regression tasks when trained on experimental datasets. Finally, we analyze the benefits and shortcomings of both categories to provide context on available identification methods and ongoing research directions.

In the optical interference scenario, due to the periodical helical phase distribution, OAM modes can interfere with the reference beams (e.g., plane or spherical waves) to extract information, and the number of fringes indicates the OAM modes, whereas the direction indicates the positive or negative states of OAM modes. As shown in Fig. 5(a), the direction of “fork-shaped” or “spiral-shaped” indicates the positive or negative states of OAM, and the fringe number indicates the topological charge. Other interference methods such as interferometers37^–44 and slits45^–47 can also realize the demodulation of OAM modes by observing the characteristics of interference fringes. The diffraction method is used to convert OAM modes into different shapes of stripe distributions through the diffraction properties of different components to realize OAM mode identification. These diffractive components include vortex grating,12^,48^–53 triangular apertures,54^–57 and lenses.58^,59 For example, Fig. 5(b) shows the experimental system of diffraction methods using Dammann vortex grating, and the corresponding far-field diffraction patterns as shown in Fig. 5(c). Due to the dependency between the topological charge of the OAM mode and the topological charge of the Dammann vortex grating, the OAM mode of the incident beam can be obtained by detecting the diffraction order of Gaussian points in the far-field pattern. In 2015, Fu et al.53 designed a 5×5 Dammann grating to detect OAM modes with a range of −24 to +24, where the target OAM modes were successfully recovered with high accuracy. In addition, the use of a cylindrical lens or embedded spherical lens for diffraction of an incident vortex beam can convert the incident light into an intensity distribution with some bright dark stripes through astigmatism transformation.60 By detecting the number of dark stripes, information about the topological charge of the OAM mode can be obtained. This also applies to the detection of vortex beams containing radial exponents.61 Zheng et al.51 proposed using a combined annular amplitude and phase grating for OAM mode identification over a wide topological charge range of −25 to +25. Their design demonstrated around 10-dB higher diffraction efficiency compared with conventional annular amplitude gratings alone. Notably, the dual grating structure was also shown to enable the detection of fractional vortex beams with topological charges between 0.1 and 1.0. In addition to conventional diffraction gratings, the detection of OAM modes can also be achieved using some special diffraction apertures. Figure 5(d) shows the diffraction results of the triangular aperture.54 In this pattern, the OAM mode can be obtained by observing the number of bright spots N on either side of the triangular array in the diffraction pattern, and the topological charge sign is related to the rotation direction of the diffraction array. Similarly, pattern detection can also be achieved using square apertures,62 binaural small holes,63 etc. The rotational Doppler effect can equate the rotation of the vortex beam to the frequency shift of the beam, and the OAM modes are further identified by measuring the beam frequency shift value.64 However, most of these schemes rely on the human eye to identify the number and direction of stripes for OAM mode identification, resulting in limited identification accuracy and mode ranges. Indeed, vortex beams are highly susceptible to phase aberrations caused by atmospheric turbulence during free-space propagation, which further reduces the identification accuracy. Especially for the hybrid OAM modes, the interference and diffraction patterns do not exhibit regular spatial stripes, which prevents OAM mode identification procedures.

For high-order or hybrid-order mode identification, the key challenges lie in how to extract mode information from the intensity distributions and establish the intensity-to-mode mapping relationship, thus achieving fast mode identification. In recent years, advances in semiconductor technology have continuously increased computing power and capabilities. This has enabled emerging applications of artificial intelligence techniques across scientific domains.65^–67 Hence, optical mode identification can benefit from the application of artificial intelligence techniques, enabling efficient identification of various mode ranges and demodulation of shift-keying communication signals through the utilization of diverse input-output data sets in a neural network model.68^–75 According to the type of neural network architecture, the demodulation of mode can be categorized as CNN-based and FNN-based technologies. The CNNs are specialized at processing high-dimensional data such as intensity images, whereas the FNNs are capable of processing low-dimensional data such as optical feature parameters. Figures 6(a) and 6(b) show the architecture and performances of seven-layer CNN for identifying OAM modes, in which the distorted intensity distribution of OAM modes was directly inputted into the model for classification and successfully constructed the OAM-SK communication link using CNN for demodulation with BER of ∼0.86%.68 Due to the same intensity distribution of vortex beams carrying conjugated OAM modes, the end-to-end mapping ability of neural networks cannot achieve accurate recognition of them. Wang et al.69 proposed a neural network scheme that combines cylindrical lenses for feature extraction. As shown in Fig. 6(c), the input vortex beam is transformed by a cylindrical lens and diffraction fringes are collected at the rear focal plane, which is then transmitted to the neural network, successfully achieving conjugate pattern recognition. Figure 6(d) shows the diagram of the OAM-SK communication link based on CNN and a cylindrical lens. The recognition process only takes about 1 ms per mode with an accuracy exceeding 99%. Similarly, using grating diffraction71^,72^,76 or interferometry methods can also transform conjugate information hidden in the phase into the intensity domain, thereby utilizing neural networks to achieve accurate decoding. Moreover, thanks to the excellent information processing ability of neural networks, they can not only recognize superimposed patterns from the collected light intensity distribution but also achieve ultrahigh-resolution recognition of fractional order patterns. In 2019, Liu et al.17 achieved superhigh-resolution identification of OAM modes using the CNN architecture, and the minimum interval between adjacent OAM modes decreased to 0.01 [Fig. 6(e)]. These fractional OAM modes were then encoded into an 8-bit grayscale image for free space transfer and then demodulated by the CNN model with BER lower than 0.02%. In 2020, Giordani et al.74 demonstrated the effectiveness of using CNNs and principal component analysis to identify and classify specific vector pattern patterns. Furthermore, the CNN model can also generate research results indicating that using neural network-based protocols to construct and characterize high-dimensional resources of quantum protocols has significant advantages. In addition to being able to identify a single OAM pattern17^,77 or CVB pattern74^,78 within a limited topological charge range, the use of neural networks can achieve multi-dimensional pattern recognition79^–83 and continuous spectrum regression.84^,85 The use of neural networks can achieve simultaneous recognition of radial exponents and OAM patterns.30 Although the CNN-based intensity-to-mode model performs well in mode identification and demodulation, the operation on high-dimensional data requires higher computational complexity, thereby resulting in slower demodulation and model construction speed. To tackle this problem, the diffractive deep neural network86 with multiple phases modulated layers embedded was implemented for the OAM mode identifications, where the mode demodulation can be performed at the speed of light, resulting in faster demodulation speed.87^,88 By utilizing progressive phase modulation, diffractive deep neural networks can also be used to achieve OAM mode spectrum analysis [Fig. 6(f)],89 with error less than 10−5.

Mode processing technologies are key components in OV-based optical communication and network, determining network complexity and facilitating practical applications in mode-division networks, which involve mode (de)multiplexing, interconnecting, and computing. This section provides a comprehensive review of these aspects, offering a systematic analysis of mode processing technologies in OV-based optical communication and network systems.

Mode multiplexing and demultiplexing refer to the processes of coupling or separating multiple OAM or CVB modes that carry independent information. Various schemes have been developed for mode (de)multiplexing, including cascaded beam splitters, angle-separated gratings, coordinate transformations, and multi-layer mode sorters, all aimed at achieving effective multi-mode coupling or separation. Furthermore, with advancements in integrated photonics, on-chip devices utilizing optical waveguides have emerged as a compact and integrated solution for (de)multiplexing OV modes.

Optical beam splitters offer a simple and convenient solution for combining multiple beams.10 A polarization-independent beam splitter evenly divides two beams from the upper and left ports into the two right ports at the bottom, achieving beam combination.10 By cascading multiple beam splitters, the number of combined beams can be further increased, such as using N beam splitters for N+1 beams.90 However, it is important to note that each beam splitter introduces a fixed 50% power loss because only one of the two output ports is typically utilized. Consequently, the basic energy utilization rate of this simple cascaded beam splitter method is limited to 1/2N. Although optical communication networks usually require only −10dBm power (under good system performance conditions) for accurate signal decoding, this approach still faces challenges in scalability due to the exponential decay of power. As mentioned earlier, the efficient sorting of multiple OAM modes can be achieved by constructing MZI arrays,37 even at the single photon level. By leveraging the principle of optical path reversibility, when the original output port is utilized as the input, the combined beam can be obtained at the original input port with nearly 100% energy efficiency. However, cascading multiple devices poses challenges to scalability. Despite these difficulties, the simple cascaded beam combiner remains the most convenient and straightforward solution for vortex beam multiplexing because the components required are easily obtainable and straightforward to operate.

A specific type of grating characterized by a forked structure is referred to as a forked grating. Initially proposed by Bazhenov,91 the utilization of fork gratings for generating vortex beams was introduced. In the experimental setup, a computer-synthesized forked grating was employed to diffract the laser beam. Observations from the experiment indicated the presence of a spiral wave with zero strength at the ±1st diffraction order of the grating within the far-field diffraction region. As shown in Fig. 5(a), due to the existence of the spiral phase wavefront, there is an obvious dislocation in the interference fringes between the OAM mode and the plane wave, which appears as a fork shape, and the number of forked branches (called the topological charge of the grating) is the same as the topological charge of the OAM mode (the TCs of OAM modes being directly proportional to the number of forks in the gratings). As shown above, according to the holographic theory, when a plane wave or vortex light is incident on a holographic grating formed by such forked interference fringes, the outgoing field will carry a vortex beam of the OAM mode corresponding to the grating topological charge, which satisfies ld=li+lf+d, where ld is the OAM mode order carried by the outgoing light field of the d’th-order diffraction. Using this correspondence between the mode topological charge and the diffraction order, we can easily deduce the order of the incident light from the diffraction order emitted in the form of the Gaussian basis mode and realize mode demultiplexing. On the contrary, when Gaussian light is incident on this type of grating at different diffraction orders, the 0’th order diffraction of the outgoing field will carry the OAM mode of ld.

The vortex grating serves as a valuable tool for the precise separation of distinct OAM states in free space through varying diffraction orders, offering the potential for parallel separation of multiple OAM beams.13^,92^–95 An illustrative example of this is the diffractive optical element that integrates a Dammann grating with a vortex phase diagram.96 By strategically constructing the Dammann vortex grating with phase transition points at 0.23191, 0.42520, and 0.52571, it becomes possible to generate seven alternative OAM states for OVs.97 Researchers have identified that through the careful design of gratings and manipulation of phase transition points, it is feasible to focus light intensity precisely on the desired diffraction orders. However, an issue arises with the imbalance in diffraction order strengths, resulting in a significant concentration of light energy on the zeroth order, potentially leading to disparities in bit error rates among OAM channels. To address this challenge and ensure consistent management of diffracted light intensity, the concept of an equalizing Dammann vortex grating has been proposed, which aims to suppress the even and zero diffraction orders.93 Despite these advancements, the detection capability of OVs remains limited. The utilization of binary phase 2D Dammann vortex gratings has shown a substantial improvement in the detection capacity of OVs as all intended diffraction orders become viable. For instance, a 5×5 Dammann vortex grating can effectively detect 25 topological charges ranging from −12 to +12,48 thereby relaxing stringent beam alignment requirements and ensuring uniform power distributions across all OAM states.98 Due to the linear superposition effect of Dammann grating, as shown in Fig. 7(a), Lei et al.13 demonstrated the independent collinear production, transmission, and simultaneous detection of OAM channels using a Dammann vortex grating, ultimately achieving a transmission capacity of 80Tbit/s with consistent power distributions across all channels. This configuration included 1600 data channels employing quadrature phase-shift keying, 80 wavelengths, two polarizations, and ten OAM states that were multiplexed, with BERs all below 10−6 at OSNR over 25 dB, as shown in Figs. 7(b) and 7(c). By combining geometric phase modulation and propagation phase modulation of the metasurface, Chen et al.14 proposed a CVB mode off-axis polarization control technology based on vector Dammann gratings and designed a mode (de)multiplexer with broadband response characteristics (from 1310 to 1625 nm) to achieve multiplexing communication of four CVB modes. By combining eight wavelengths and two polarizations, 1.56Tbit/s quadrature phase shift keying (QPSK) signals were successfully transmitted with BERs less than 10−6. In addition, based on the polarization control ability of the metasurface, it can also achieve joint multiplexing and demultiplexing of polarization and OAM modes.94^,99^–102 Similarly, the use of metasurfaces can also achieve joint analysis of input beam OAM modes and polarization states. Usually, the abovementioned demultiplexing devices can only respond to specific polarization states or operating wavelengths,103^–106 thus limiting the dimensions of multiplexing. To address this issue, Gao et al.107 proposed a multi-optical parametric demultiplexing device based on a single-layer dielectric type metasurface, including wavelength, polarization, and OAM mode. It can demultiplex the three-dimensional optical parameters of the incident light and focus them at different spatial positions on the focal plane, greatly simplifying the complexity of traditional multidimensional multiplexing systems.

By leveraging the correspondence between the diffraction order and the topological charge of the vortex grating, it becomes possible to achieve off-axis control and demultiplexing of OAM modes. However, the inherent Bragg diffraction order of the grating introduces a spectroscopic response, limiting the upper bound of energy utilization efficiency to 1/N. Inspired by the focusing characteristics of the radial quadratic phase on the plane wave in the Cartesian coordinate system, a corner lens with angular quadratic phase distribution has been proposed.108^–110 When an OAM mode with an azimuthal phase gradient (proportional to the azimuthal angle) is incident on the corner lens, it is focused to different angular positions,108 as depicted in Fig. 8(a). The angular position of the focused spot corresponds to the topological charge of the incident mode, allowing for the calculation of its topological charge by detecting the angular position of the spot. Experimental results demonstrate that the corner lens efficiently sorts OAM modes with topological charges ranging from −84 to +84 with an interval of Δl=6. Furthermore, by combining the geometric phase and dynamic phase of the subwavelength structural unit of a metasurface with the independent control ability of orthogonal circular polarization, it becomes possible to achieve simultaneous sorting of the SAM and OAM modes of photons using a single spin-decoupled metasurface.99Figure 8(b) illustrates that for LCP incident light (σ=1), different OAM modes are mapped to the upper half of the ring, whereas the RCP incident light exhibits the opposite behavior (σ=−1). This approach enables the simultaneous sorting of SAM and OAM modes as well as vector OVs, such as CVB modes. However, the sharp focus shape of the spot introduces significant crosstalk between adjacent modes, limiting the achievable minimum topological charge interval. To alleviate this issue and improve spot separation, a radial secondary phase inversely proportional to the input wave radius can be introduced to form a radial-angular hybrid lens.111 This effectively reduces the crosstalk level, achieving 4% inter-mode crosstalk when the topological charge interval is 3. By combining the angular lens phase with the superoscillatory phase, the angular superoscillatory phenomenon can be utilized to map OAM modes with adjacent topological charges into focused modes with a narrow half-width at half-maximum, thereby scaling up the resolution by ∼3.2 times compared with a normal angular lens.112 Although ultra-wide range OAM mode sorting (up to |l|=250) can be achieved by optimizing the angular lens parameters,113 the maximum number of identifiable OAM modes in a single shot is still limited to around 60 due to the tradeoff between inter-mode crosstalk and mode spacing caused by confining the selected mode space to a single circle in the focal plane. To overcome this tradeoff, monolithic spiral metalens has been developed,114 which focuses the incident light beam into a multiple-turn spiral ring on the focal plane according to its topological charge. This strategy enables the successful sorting of mode orders ranging from −250 to +250, with a mode interval of 3 and a maximum number of identifiable modes in a single shot reaching 168.114 Furthermore, when combined with the orthogonal polarization independent control capability of the metasurface, it enables the sorting of the full angular momentum state.

In 2010, Berkhout et al.115 proposed an optical coordinate transformation technique for OAM mode, which utilizes two-phase modulation surfaces to convert the input light field from Cartesian coordinates to logarithmic polar coordinates. Through this transformation, due to the different phase periods of different OAM modes along the angular direction, the different OAM states in the incident vortex beam will be transformed into a set of plane waves with different transverse phase gradients, which will be focused by the lens and separated in spatial position.115^,116 However, this method often suffers from mode crosstalk caused by overlapping light spots. To solve this problem, coordinate transformation methods combined with beam copying117 and spiral transformation methods118^,119 have been proposed to improve the separation of OAM modes in the output plane. Their core lies in the number of periods when expanding the OAM mode into a transverse phase gradient so that when focused by the lens, they have a smaller focused spot. In addition, the accuracy of coordinate transformation heavily depends on the precise alignment of the two-phase modulation planes. To solve this problem, as shown in Fig. 9(b), Wen et al.120 processed unwrapped and phase directors on both sides of a thick glass, proposed a compact mode coordinate transformation system, and implemented 50-km OAM fiber communication [Fig. 9(c)]. The 16-Gbaud QPSK signals with an aggregate capacity of 2.56Tbit/s, a spectral efficiency of 10.24bit/(s·Hz), were successfully transmitted with BERs below 10−6. Lightman et al.121 utilized 3D laser printing technology to achieve direct integration of two layers of cursor phase transformation, and demonstrated micrometer-level OAM mode multiplexing devices. This provides new ideas for the implementation of new OAM mode control devices.

Using Pancharatnam-Berry (PB) phase optical elements, such as geometric phase-based metasurfaces or liquid crystals, to load the unfolded phase and corrected phase of the vortex beam coordinate transformation, the conjugate response characteristics of the PB phase to orthogonal circularly polarized light can be used to achieve demultiplexing of vector vortex beams,122^–128 such as CVB modes. When the CVB mode is incident on the PB phase-based geometric transformation optical element, the conjugate OAM modes under the LCP and RCP components can be converted into two spatially separated plane waves propagating in the same direction and guided to the same spatial position representing the polarization topological order of the incident vector beam.122 Thanks to the wide-band phase response characteristics of the PB phase, it can efficiently classify CVB in a wide band. Similarly, when the unfolded and corrected phases of the spiral transformation are loaded or combined with diffractive splitting and fan-out mechanisms, the period of the plane wave can be extended and the resolution efficiency of CVB mode sorting can be further improved and crosstalk can be reduced. Using a vector metasurface or liquid crystal composed of elliptical silicon nanopillars to load the phase of the spiral transformation, the LCP and RCP resolution of CVB can be transformed to achieve efficient and low-crosstalk CVB mode demultiplexing.129 In addition, it is worth mentioning that by combining the spiral transformation with the PB metasurface, in addition to being able to achieve the sorting of CVB modes, it is also suitable for the simultaneous sorting of SAM and OAM.130

Although the mode demultiplexing scheme based on coordinate transformation has made significant progress in terms of compactness and high resolution, challenges remain in expanding the number of OAM or CVB modes involved in the multiplexing and demultiplexing processes. In addition, there are limitations in achieving flexible sorting functions due to the constraints of fixed transformation directions. To address these challenges, researchers have introduced mode sorting schemes based on multi-layer modulation.87^,88^,131^–136 Techniques such as multi-plane light conversion (MPLC) and deep diffractive neural network (D2NN) facilitate the step-by-step transformation of the light field through a multi-layer phase or amplitude modulation. This approach enables arbitrary linear transformations from the input light field to the output light field, particularly unitary transformations that are well-suited for processing orthogonal optical modes. In a noteworthy achievement by Fontaine et al.132 in 2019, 325 LG modes were successfully spatially separated using an MPLC setup consisting of seven phase plates. A single SLM and a reflective mirror were used to construct a compact mode sorter, as depicted in Fig. 10(a). The phase distribution of the MPLC’s phase planes is typically determined using a wavefront matching algorithm, which iteratively adjusts the phase of each pair of input and output modes on each phase plane to satisfy the desired input–output transformation relationship. Coaxial OAM modes are converted to Gaussian base modes with different spatial spacing, allowing easy coupling to single-mode fiber (SMF) arrays, as shown in Fig. 10(a). Similarly, SMF arrays can be used for multi-mode multiplexing operations. Based on this configuration, 210 azimuthal and radial components of LG modes are successfully sorted with insertion losses (ILs) and crosstalks below −8dB, as shown in Fig. 10(b). Utilizing an 11-port SMF array, which, before five phase plane modulations, converts the Gaussian mode inputs into 11 coaxial OAM modes ranging from −5 to +5,13710Gbit/s on-off keying signals were transmitted in a 5-km few-mode fiber (FMF) and decoded with BER achieving 10−7. The mode crosstalk of the multiplexing and demultiplexing processes is less than −20 and −10dB, respectively. An error tolerance analysis of MPLC provides a comprehensive design guide for mode (de)multiplexers, reducing fabrication and experimental demonstration errors by properly setting parameters such as input/output beam waists, beam array spacing, and inter-phase plane propagation distance.138 Reflective OAM mode sorters, which require tilting the incident beam and exhibit poor stability, can mitigate errors by fabricating transmissive MPLC on glass plates. Although the performance of MPLC can be further improved by optimizing parameters such as phase plane spacing and mode size using algorithms such as particle swarm search,139 the lack of a strong goal-oriented optimization method often leads to optimization stagnation due to the convergence to local optimal solutions. To address these challenges, diffractive optical neural networks, which have garnered attention in recent years, have been employed for the classification of vortex beams. Each layer within the diffractive optical network functions as an amplitude/phase-modulating plate. Zhao et al.87 demonstrated the use of a three-layer optical network in 2019 to detect topological charges ranging from 1 to 10. In 2021, Huang et al.88 implemented a five-layer optical network with complex amplitude modulation to achieve all-optical processing of vortex beams. This comprehensive approach encompasses the generation, detection, and transformation of vortex beams. The researchers also theoretically designed vortex beam keying, multiplexing, and demultiplexing devices based on this methodology, introducing novel perspectives to signal processing. For example, using four layers of complex amplitude modulation and phase modulation, they achieved the coupling and separation of four OAM modes134 and the displacement control of 11 OAM modes.140

However, there is still a lack of integrated parallel wavelength division multiplexing (WDM) devices specifically tailored for wavelength-mode hybrid multiplexing. In an effort to enhance performance and efficiency, a 2×8 polymer parallel arrayed waveguide grating has been designed and fabricated for an MPLC LP-mode multiplexer operating at four wavelengths in the O band, simultaneously multiplexing to two output waveguides.141 This concept can be extended to support more wavelengths and introduce more spatial modes, enabling high-capacity data transmission. However, the practical performance of these systems is impacted by the number of required phase planes, particularly in terms of mode conversion efficiency. For symmetrical modes such as Hermit-Gaussian modes, introducing a symmetrical boundary condition can greatly reduce the number of necessary phase planes.132 Another challenge arises from the wavelength dependence of wavefront modulation, which poses difficulties due to the monochromatic coherent nature of the design. Overcoming this challenge and achieving efficient and broadband achromatic operation in multiplexing/demultiplexing devices are crucial for hybrid optical multiplexing utilizing WDM and mode division multiplexing (MDM). To address these issues, a general achromatic broadband multilayer diffraction model is proposed to compensate for dispersion by aligning the wavefronts of discrete multi-wavelength light fields. This model facilitates the design of achromatic OAM beam generators and achromatic linear polarization mode multiplexers for coarse WDM optical interconnects,142 as depicted in Fig. 11(a). The transmission of a signal composed of six wavelengths × 3 modes at 10Gbit/s was successfully achieved with BER less than 10−6. By incorporating the wavelength-dependent characteristics of interlayer diffraction or leveraging the polarization-independent control capability of metasurfaces, the multi-layer mode sorter can further expand the working bandwidth or achieve simultaneous multiplexing and demultiplexing of multiple physical dimensions, surpassing the capabilities of OAM mode and CVB mode. Utilizing the polarization separation effect of the prism combined with a single SLM enables the sorting of up to 120 modes of vector vortex beams in the visible light band.143 Furthermore, by utilizing the orthogonal polarization-independent control capability of the metasurface to construct a polarization optical diffraction neural network, sorting, multiplexing, and demultiplexing of 14 orthogonal polarization vortex beams are successfully demonstrated.144 Moreover, more generally, the diffraction neural network based on the vector-controlled metasurface can easily sort arbitrary orthogonal vector beams, beyond vortex beams.145 Furthermore, Fig. 11(b) demonstrates that by utilizing the wavelength-sensitive characteristics of interlayer diffraction and introducing a multi-wavelength discrete wavefront matching algorithm, a five-layer metasurface can achieve joint multiplexing and demultiplexing of three CVB modes and three wavelengths.146 This approach provides an effective means for multi-dimensional multiplexing communication.

The development of cost-effective integration technology is likely to be critical for the future implementation of MDM communication. This involves exploring various structural designs for integrated devices utilized in OAM-based communications.147^–156 An approach for generating an adjustable OAM beam involves a construction based on a ring resonator. The ring-resonator-based OAM emitter comprises a micro-ring resonator and angular grating structures with periodic modulation of the refractive index in the azimuthal direction. The construction of the ring-resonator-based OAM emitter is similar to that of a straight waveguide’s grating coupler.157 Angular gratings are incorporated into the ring resonator design to extract the OAM modes from the micro-ring waveguide and release them into free space. When the wavefront twists in the azimuthal direction, the output light transforms into an OAM-carrying beam. Various ring-resonator-based structures efficiently produce and detect numerous OAM beams. For instance, a chip-to-chip OAM-multiplexing communication link has been demonstrated using a pair of circular phase arrays, with each carrying a 20-Gbit/s QPSK signal,158 as shown in Figs. 12(a) and 12(b). The use of multi-mode ring resonator-based OAM emitters has demonstrated a chip-to-chip four-OAM-multiplexed connection, each carrying a 16-Gbit/s QPSK signal with a BER achievable within the 3.8×10−3 FEC limit,150 as shown in Figs. 12(c) and 12(d). With this setup, it is theoretically possible to concurrently tune the OAM ordering of the four generated OAM beams at the same wavelength. In contrast to the ring-resonator-based OAM emitter, the wavelength spectrum of the OAM beams generated by the circular phase array device is continuous, which may be compatible with a wavelength-division multiplexing system. In addition, a chip-to-fiber OAM-multiplexing communication link with three OAM beams (ℓ=+5,+6,+7⁠) has also been demonstrated using the circular phase array.159 However, the relatively large footprint of this device may limit its wide-scale integration.

Another method for achieving broadband OAM production using a reasonably small construction is to utilize a specially built subwavelength optical antenna. Reports have outlined several design principles, such as combined phase control of the optical path and local resonance, and overlaid binary fork gratings. A subwavelength surface structure, such as superposed holographic fork gratings, has been placed on top of a silicon waveguide to create the silicon OAM chip.151 This structure facilitates highly efficient and pure polarization diversity OAM modes (x-/y-polarized OAM+1/OAM−1) in the broadband range of 1500 to 1630 nm. A second technique for creating a subwavelength optical antenna relies on local resonances in a specially built structure and combined phase control of the optical channel. The broadband characteristic of this device may stem from the local resonator’s comparatively low Q factor (<15).149 However, directly detecting the phase information of high-speed modulated light without the need for an external, local oscillator for reference can be challenging and costly for short-range optical communications, such as those in data centers. Finally, the spin-orbit interactions in Si nanodisks coupled to a network of single-mode Si waveguides on a silicon-on-insulator substrate are used for quadrature phase-shift keying, eight phase-shift keying, and 16-ary quadrature amplitude modulation signals using integrated on-chip high-speed coherent signal demodulation160 [see Figs. 12(e) and 12(f)].

In MDM optical networks, establishing interconnections between different channels is essential for enhancing network flexibility. Depending on the application scenario, mode interconnection can be categorized into mode switching, mode drop/add, mode routing, and mode multicast. This section reviews the advancements in these interconnection technologies.

Mode exchanging, which is the exchange of information on different channels through independent transformation of coaxial transmission multi-modes, is crucial for information processing at the node. The use of a reflective spiral phase can simply and directly realize the exchange of two OAM modes. Figure 13(a) shows the exchange of two coaxially transmitted OAM mode channels, where the topological charges of the incident beams are l=+8 and l=+6, respectively.10 After modulation by the spiral phase plate with l=−14, the l=+8 channel will be converted to l=−6, and the l=+6 channel will be converted to l=−8. It should be noted that the spiral phase plate is loaded on the reflective spatial light modulator, and the topological charge of the output beam will be reversed after reflection. Therefore, l=−6 and l=−8 are converted to l=+6 and l=+8, realizing the exchange of two channels. Furthermore, by cascading 2n+2 reflective spatial light modulators, arbitrary and independent mode conversion of n OAM modes can be realized.161 Based on this scheme, selective data switching among three 100Gb/s QPSK signals was achieved with a 2.1-dB optical SNR penalty. Later, in 2019, He et al.162 extended this modulation principle to the CVB beams using Pancharatnam-Berry phase q-plates, which can impose opposite helical phase distributions to both left-handed and right-handed circular polarizations, enabling the exchange of not only OAM modes with different polarization but also CVB mode. They successfully achieved mode exchange, retrieving, and conversion of mode channels with BERs approaching 10−6.Figure 13(b) shows the exchange diagram of two CVB modes of −1 and +3. As a proof of concept, 50-Gbit/s QPSK signals carried by these two channels are successfully exchanged and the BERs of digital signals after demodulation are lower than 10−6 as the received power increases to −20dBm.

To realize the cross-connection of M coaxially transmitted mode channels, it requires at least M−1 helical phase plates for these conversions. Functionally separated phase plates play a critical role in enhancing the modulation capabilities for mode channel cross-connections. In 2016, Liu et al.163 demonstrated the reconfigurable joint OAM modes and space switching using functionally designed phase distributions loaded on the spatial light modulator, achieving the mode exchange and space switching through the combination of helical phase plates, fork-shaped gratings, and gradient phases, which can be integrated transmission, exchange, and space switching functions in single optical elements. Another strategy for coaxial OAM mode cross-connection using the radius independency of OAM modes with different topological charges converts the target OAM mode channels back to Gaussian beams at the centers and then uses a beam splitter and aperture for filtering. This is usually realized by the cascaded modulation, which exchanges OAM modes at different output planes.164 However, these techniques heavily rely on the mapping relationship between input and output OAM mode vectors, and the selection of OAM mode vectors should be properly considered for establishing the mapping relationship, which could not realize arbitrary conversion of OAM modes for complicated mode cross-connections. In 2020, Brandt et al.165 proposed an OAM mode based on a high-dimensional quantum gate, where the OAM modes are selected as quantum states for operations. In this scenario, the MPLC optical system was employed for establishing the arbitrary mapping relationship of OAM modes, enabling the cross-connection of OAM mode vectors. Furthermore, Huang et al.88 proposed a five-layer diffractive deep neural network for the cross-connection of OAM modes, where the mode processing task is more complicated with a maximum mode distance of 3. They also verified the possibility of a diffractive deep neural network in all-optical signal processing, and the measured BER values for six input OAM mode channels below 10−4 in simulation. Moreover, these exchanges of OAM modes can also be realized by optical parametric oscillator166 and integrated vortex emitters167 for on-chip mode conversion or generation. The 3D manufacturing technologies based on two-photon polymerizations and femtosecond laser direct writings have recently provided new possibilities for mode control devices due to their excellent device-shaping capabilities. It can not only fabricate photonic lantern-type mode multiplexers with low insertion loss and low mode crosstalk168 but also achieve compact mode conversion by 3D stacking multiple phase-modulation surfaces.169

For the coaxially transmitted multiplexed channels, the adding and dropping of target channels are necessary when promoting the mode-division network to practical applications. This necessitates the independent modulations to multiplexed mode channels, giving each incident mode independent phase or spatial location modulations. In 2014, Ahmed et al. proposed an OAM mode add/drop multiplexer, involving three steps for the conversion,20^,170 as shown in Fig. 14(a). First, down-convert the target OAM modes channels back to Gaussian beams located at the centers, whereas other multiplexed beams remain ring-shaped vortex. Second, drop the center Gaussian beam and add another beam that carries information. Note that, this beam can be either a Gaussian beam or an OAM beam, which should possess different topological charges compared with the multiplexed channels. Finally, upconvert the multiplexed channels back to their original topological charge. It is noted that for supporting more complicated adding and dropping functions, reconfigurable phase modulation elements are necessary for practical applications. Huang et al. designed a liquid-crystal-on-silicon-based optical OAM mode add/drop multiplexer, demonstrating the adding/dropping of single OAM modes from three multiplexed OAM beams,171 as shown in Fig. 14(b). To better separate the dropping or adding beams, a phase hologram grating was employed for imposing diffraction angle to incident beams. As shown in Fig. 14(c) experimental results show that for the multiplexed OAM beams carrying 100Gbit/s QPSK data, the power penalty of less than 2 dB is observed with a BER value of 2×10−3. These OAM modes adding and dropping technologies offer flexibility, and they inevitably disrupt the helical wavefront of OAM modes during up and down conversion. In addition, coaxial conversion of OAM modes can introduce significant mode crosstalk, degrading the performance of optical communication systems. By contrast, imposing diffraction angles using gradient phase elements allows for channel adding and dropping while preserving the helical distribution of incident OAM modes, thereby mitigating these drawbacks. In 2022, He et al.19 proposed a polarization-dependent gradient phase modulation strategy and fabricated a local polarization-matched metasurface to add/drop polarization multiplexed CVBs. This metasurface enables the independent modulation to two orthogonal bases; in this scenario, the radial- and azimuthal-polarized CVB modes and the corresponding gradient phase with target diffraction angle were designed for the azimuthal polarization, whereas the blank phase with reflection was designed for radial polarization. The normally incident azimuthal-polarized CVB modes were dropped, and the obliquely incident azimuthal-polarized CVB was added for transmission, whereas the radial-polarized was reflected for coaxial transmission. As a proof of concept, three polarization multiplexed CVBs, carrying 150-Gbit/s QPSK signals, are successfully added and dropped, and the BERs approach 1×10−6. To realize more functional mode channel adding/dropping, it is crucial to selectively manipulate OAM modes and their spatial location via a single diffractive element. Therefore, Xiong et al.172 proposed a multi-layer diffractive neural network-based OAM mode adding/dropping multiplexer, where the incident plane was separated into multiple regions for different functions. The adding OAM modes after modulation will be shifted to the through port, whereas the dropping channel will be shifted to the target location. Results show that four OAM mode channels ranging from l=+1 to l=+4 can be selectively added and dropped with diffraction efficiency and mode purities exceeding 95%. More importantly, this multiplexer can be cascaded and implemented for additional modulations, enhancing the flexibility of this add/drop multiplexer.

Routing mode channels among transmission ports is crucial for future scalable optical communication networks, facilitating the interconnect and exchange of data among multiple OAM channels. This necessitates the generation of single or multiple OAM modes to different spatial locations that match the received ports. In 2013, Yan et al. designed sliced phase patterns for OAM mode multicasting, and by optimizing the design of phase pattern, the power of multicasted OAM channels can be well equalized.173 Fu et al. and Meng et al. have also proposed similar research using grating-based multicasting methods, enabling multicasting of up to 40 OAM modes and implementation in the encoding of OAM-SK signals. For the implementation of OAM data-carrying communication, Let et al.174 designed an OAM router based on a Dammann vortex grating and generated a 7×7 OAM modes array for OAM mode channel multicasting. The multiplexed channels with different l are routed simultaneously and recovered to Gaussian beams at different diffraction orders corresponding to the gratings’ topological charge. By selecting appropriate OAM modes, flexible multicasting of channels is achieved. In addition, the fiber mode coupling mechanism can also realize multi-OAM mode routing on an all-fiber platform.175

The orthogonal and linear characteristics of OAM and CVB modes make them ideal for low-latency, high-parallel all-optical computation, paving the way for advanced all-optical signal processing technologies. In this section, we review recent advancements in mode optical computation, focusing on mode multiplication and division technologies, mode logic operations, and mode filtering techniques.

The geometric transformation optics enable efficient implementation of mode multiplication and division operations.176^–180 These operations manipulate helical wavefronts through three steps: decomposition of the input azimuthal phase gradient into a linear phase gradient, creation of multiple copies, and wrapping of the extended phase gradient distributions into a doughnut shape. Mode division operations achieve a similar outcome by utilizing a mask to select a fraction of the linear phase gradient in the input, replacing the copied light fields. In 2016, Zhao et al.176 employed linear optical transformation between log-polar and Cartesian coordinates to successfully multiply and divide OAM modes from l to 2l and 2l to l, respectively, and the concept of mode division and multiplication using coordinate transformation is shown in Fig. 15(a). The experimental system for verifying the proposed transformation involves three beam splitters, three SLMs, and two lenses for loading and compensating the distortions. The simulated and experimental results show that the mode purities measured for mode halving and doubling OAM states can reach beyond 87% and 40%, respectively. However, this method requires multiple optical elements and avoidably introduces energy losses during beam copying and filtering. In 2019, Ruffato et al.178 proposed a novel mode multiplication and division method with diffractive transformation optics, which basically preserves axial symmetry and avoids the limitation of conventional log-polar coordinate transformation methods. By combining multiple circular-sector transformations onto a single optical element, multiplication of the input OAM can be achieved by mapping its phase onto complementary circular sectors. Conversely, by combining multiple inverse transformations, different complementary sectors of the input beam can be mapped into an equal number of circular phase gradients, achieving a division of the initial OAM. This approach allows for the multiplication and division of OAM modes in a more compact manner, reducing the number of optical operations and optical elements and finally increasing the optical efficiency. Later, Wen et al.181 proposed arbitrary multiplication and division of OAM modes using the azimuth-scaling spiral transformation, overcoming the integer factors requirement and complex beam-copying and multi-transformation diffraction stages, as shown in Figs. 15(b) and 15(c). The OAM mode multiplication can also be achieved through Fermat’s spiral transformation182 [Fig. 15(d)], circular-sector transformation,179 and plasmonic vortex cavities.183 Recently, Wang et al.184 successfully achieved the multiplicative operation of four CVB modes with multiplier factors N=+2 and N=−3 and validated its feasibility in the CVB channel switching with BERs below 10−6, using 3D-printed components realized through femtosecond laser lithography.

Optical logic operations are crucial for optical digital computation, offering general-purpose calculation capabilities with high speed, low crosstalk, and high throughput. Traditionally, optical logic states have been defined by linear momentums distinguished by intensity distributions. However, this approach suffers from blurred discrimination boundaries, limiting its long-term applicability. In 2020, Brandt et al.165 proposed and experimentally demonstrated high-dimensional quantum gates using OAM modes and radial index. As shown in Fig. 16(a), by utilizing a few-layer phase modulation plane, a multi-plane optical conversion system was successfully constructed and multiple logic gates including Pauli gates and Hadamard gates were implemented with an average visibility of over 90%. For the realization of general-purpose optical computation, it is essential to implement OAM modes for fundamental logic operations, including “AND,” “NOT,” and “OR” operations. In 2021, Wang et al.21 introduced an OAM mode logical operation implemented through optical diffractive neural networks, as shown in Fig. 16. By leveraging OAM modes as logic states, this approach achieves enhanced parallel processing capabilities, improved logic distinction, and increased robustness of logical gates. This approach capitalizes on the inherent infinity and orthogonality of OAM modes. Results show that the few-layer optical diffractive neural network successfully implements the logical operations of AND, OR, NOT, NAND, and NOR in simulations, satisfying the fundamental requirements in optical computations. The linear non-separable operations of XNOR and XOR were obtained through cascading multiple basic logic operations, and half-adder gates were also constituted through these operations. This strategy provides a new avenue for optical logic operations and is expected to promote practical application in optical computing.

Wavelength filtering technologies are crucial in wavelength-division multiplexing communication, where filters are employed to remove out-of-band noise from multiplexed channels. For OAM modes, which possess helical phase wavefronts, the phase distributions are more susceptible to distortion and interference during transmission. This leads to power from a single mode being randomly distributed to multiple OAM modes, a phenomenon known as mode crosstalk. Consequently, there is an urgent need for OAM mode filtering technologies that enable efficient mode extraction and suppression with a wide modulation range. These technologies should facilitate the flexible extraction of target mode channels while effectively suppressing irrelevant channels. Figure 17(a) shows the concept of a tunable OAM mode filter, and four multiplexed OAM modes are inputted into the tunable OAM mode filter, which can be either transmission or suppression for the selected OAM mode channels.185 In 2014, Huang et al.185 proposed a tunable mode filter for spatially multiplexed OAM beams using programmable SLMs as an optical geometric transformation-based OAM mode sorter. As shown in Figs. 17(b) and 17(c), the forward pass maps the multiplexed OAM beams at the input plane into spatially overlapped rectangular plane waves, each tilted proportionally to the vortex charge of the corresponding input OAM beam. A lens focuses these tilted plane waves into spatially distinct elongated spots at its focal plane, with an inter-spot separation of λf/2πa (where λ is the wavelength and f is the focal length). In the reverse pass, the beams are reflected by a mirror at the focal plane and pass through the mode sorter again. The mirror collimates the reflected elongated beams, converting them to rectangular plane waves with different tilts. The geometric transformer then performs an inverse transformation, converting the tilted plane waves back into vortex beams with a ring-shaped intensity. Replacing the mirror with a programmable mirror array allows selective control over the passage or blocking of each OAM mode, achieving tunable OAM mode filtering. By integrating spiral and lens phase modulations, Wu et al.186 utilized depth-dependent transformations and successfully achieved tunable filtering of five OAM modes with crosstalk below −10.9dB and BERs below 10−6.

An azimuthal beam shaper has been proposed by Lin et al.,187 which is capable of OAM mode filtering in azimuthal direction by modulating in its Fourier domain, the OAM mode spectrum. As shown in Fig. 17(d), such an OAM mode filter utilizes MPLC technology and consists only of a spatial light modulator and a mirror. It works in a sequence of “OAM mode demultiplexing—mode spectral modulation—OAM mode multiplexing,” and the complex amplitude of the mode spectrum can be flexibly modulated. The linear phase (i.e., the grating) is added to invert the optical axis of the beam and is applied to all OAM modes. If the linear phase for some OAM modes is eliminated, these OAM modes would exit the other port and dissipate. Therefore, these orders would be removed from the output OAM mode spectrum. Figure 17(d) shows the case that the gratings are preserved only for even-order OAM modes (l=2k, k∈Z), and the odd-OAM modes (l=2k+1, k∈Z) are filtered out. Due to the widening of the spectral spacing, the recombined output beam forms a two-petal shape. Although the output three-petal intensity distribution is similar to that of the three-fold Talbot self-imaging, the self-imaging requires phase modulation only and is thereafter lossless, whereas the filtering process is always accompanied by an energy loss. Although some modes are dropped from the region where the grating is eliminated, the modes can be added by being incident in the same region with an inverse optical axis. In this way, the azimuthal beam shaper acts as an add-drop multiplexer for the OAM-MDM communication system. In addition, by combining polarization control elements and forming a quasi-symmetric unit, as shown in Fig. 17(c), filtering of the total angular momentum of photons consisting of SAM and OAM can be achieved, and experimental results show that 42 total angular momentum states can be effectively controlled.188

In the free-space optical transmission of optical vortex beams, the divergence of beam radius189^–191 and disturbance of atmospheric turbulence192^–195 become important factors that affect the stability of transmitted signals. For example, when vortex beams propagate over a long distance in free space, their spot size can reach over 10 m196 and the spot will be significantly distorted by turbulence.22^,190 The girdle radius and beam divergence of vortex beams increase with the value of TC.197 For an OAM mode with l and p=0, the divergence angle αl can be derived as198αl=|l|+122kw0.(3)

Therefore, the waist radius will expand during forward transmission, and the larger the value of l, the greater the divergence, resulting in a decrease in the optical received power and thus affecting the communication quality28 [Fig. 18(a)]. These issues can be alleviated by designing lens groups to converge the intensity of light beams, improving the communication intensity utilization. By properly designing the transmission lenses and constructing an OAM-multiplexed free-space communication system shown in Fig. 18(b), Allen et al.189 successfully received a power of 15 dB for high-order OAM modes and reduced the modal crosstalk by 10 dB. This approach not only reduces power loss in finite-aperture FSO links based on OAM modes but also enhances the system’s robustness against angular errors, although it may decrease tolerance to displacement. In addition, the received power can also be improved by designing diffraction-free structured beams within a certain diffraction distance. At the same time, to ensure the number of channels for OAM mode multiplexing, it is also necessary to regulate the beam waist radius of the beam. The perfect vortex beams, originating from a kind of Bessel beam set, can realize diffraction-free propagation and the topological charge can be independent of the beam waist radius, which provides a solution to generate high-order vortex beams. By leveraging the binary amplitude modulation and paraxial Gaussian approximation, Chen et al.199 have generated a perfect vortex beam with an OAM order of 90 and a waist radius of 1 mm. Besides, with the coherent light combination principle, perfect vector vortex beams were generated using radial phase-locked Gaussian arrays.200 By controlling the radial phase distribution of traditional OAM beams, a quasi-ring Airy vortex beam can also be generated to improve the received power and achieve better BER performance under limited receiving aperture conditions,191 as shown in Fig. 18(c). However, the above studies on perfect vortex beams only focused on the optical field modulations and have not yet been applied to optical communication. These schemes can result in a smaller mode field distribution for the beam after long-distance transmission, which helps increase received power and reduce the impact of turbulence disturbances. However, this may raise the alignment precision requirements between the transmitter and receiver, including relative lateral displacement and angular errors, because a smaller mode field distribution is associated with smaller characteristic scales. Misalignment could lead to OAM spectrum spreading and power coupling into other modes. Therefore, although increasing received power, further consideration is still needed to ensure the system’s robustness against external disturbances, such as turbulence and component misalignment, to enhance the communication quality and stability of optical communication networks.

Atmospheric turbulence poses another major challenge for decreasing the quality of optical communications using optical vortex beams. It arises from random refractive index fluctuation in the atmosphere, impacting applications such as astronomy, satellite links, and laser beams. Distortions from turbulence can twist and deform the helical wavefront,23 degrading channel performance for OAM-based communication systems. OAM beam propagation through atmospheric turbulence can be simulated using the Hill-Andrew model or Kolmogorov model.23^,201 This model converts turbulence perturbations into the corresponding phase distortions imparted on vortex beams. Phase distortion is characterized by turbulence intensity and propagation distance parameters. As shown in Fig. 19(a), when pure OAM mode passes through atmospheric turbulence, its intensity distribution will undergo significant distortion, accompanied by power redistribution, which will introduce intermodal crosstalk and reduce the performance of the communication system.193 For the compensation of atmospheric turbulence, there are two key strategies for compensating the phase distortion of turbulence, including adaptive optics system192^,194^,202^–206 and deep learning-assisted technologies.23^,201^,205^,207 The adaptive optics (AO) system is the most used turbulence aberration compensation scheme, which mainly consists of a wavefront detector, a reference Gaussian light, and a wavefront.192 The distorted vortex beams together with the wavefront detector and reference beams are inputted into the feedback controller, and the corresponding compensation phase is calculated by an iterative algorithm and compensated at the back end. Researchers have proposed the AO systems shown in Figs. 19(b) and 19(c), which successfully compensated for the turbulence of OAM mode channels and improved the communication quality.192^,202 In addition, by utilizing the Gerchberg-Saxton (GS) algorithm, Fu et al.208 proposed a non-probe OAM mode compensation scheme that does not require a reference beam, which can effectively alleviate the complex optical path configuration required for adaptive optics schemes. However, this approach suffers from large computational delays due to the requirement of multiple wavefront detectors and correctors. Moreover, due to the lack of learning and data-storing ability during the computational process, such algorithms are prone to converge to a local optimum, resulting in a relatively low compensation accuracy.

By leveraging the convolutional and pooling operations in the CNN model, the computational speed for turbulence compensation of OAM modes can be significantly improved.23^,205^,207^,209^,210 This algorithm enables faster compensation speed compared with adaptive optics systems while decreasing the system complexity required for pre- or post-compensation. For example, Liu et al.23 proposed a CNN architecture to predict turbulence-induced phase distortions for compensation purposes. The CNN model took the intensity distributions of an undisturbed reference beam and a distorted test beam as input. During training, the CNN learned to extract and compare turbulence-related features between the beams. This approach operates on a similar principle to adaptive optics systems, which also aim to sense phase aberrations by analyzing intensity variations introduced. Results demonstrated the mode purities of an aberrant vortex beam with a turbulence intensity of 10 to 13 from 39.52% to 98.34%. Later, Xiong et al. experimentally constructed the compensation system for OAM mode multiplexing optical communication, and the bit-error-rates were decreased by three orders of magnitude and the crosstalk was reduced from −23.15 to −29.46dB,207 as shown in Fig. 19(d). Later, an improved neural network architecture of generative adversarial networks was proposed by Zhan et al.211 for compensating vortex beams in oceanic turbulence to achieve higher accuracy with minimal data required. This generative adversarial network (GAN) model consists of two main components: a generator and a discriminator. The generator aims to capture the true data distribution and generate synthetic images similar to real examples. The discriminator evaluates the authenticity of images, estimating the probability that an input image originates from the real training data distribution or is synthesized by the generator. Overall, these proposed schemes offer a robust approach for mitigating a wide range of atmospheric turbulence effects. By leveraging its ability to both rapidly sense aberrations and accurately compensate perturbations in neural networks, it provides an efficient strategy for counteracting real-time fluctuations introduced by turbulent channels. Moreover, by utilizing diffractive neural networks,212 it is also possible to achieve all-optical compensation of OAM beams.

Compared with existing technologies such as WDM and PDM, the core advantage of OAM systems lies in their theoretically infinite orthogonal modes, which provide additional degrees of freedom for high-dimensional multiplexing in signal modulation and encoding, significantly enhancing channel capacity and spectral efficiency. Although different OAM modes can be generated and controlled using a single laser and simple optical components to construct transmitters, multiplexers, and receivers, this demonstrates some cost advantages in terms of equipment for commercialization. However, due to mode-related beam divergence and sensitivity to disturbances such as turbulence, OAM communication systems still face challenges in transmission distance and interference resistance compared with fiber-based WDM systems, along with substantial obstacles in deployment and maintenance costs. OAM communication systems may encounter issues such as power attenuation and inter-modal crosstalk in long-distance and turbulent environments, leading to higher error rates and poorer fault tolerance. As a result, most free-space communication demonstrations currently focus on short distances or specific turbulence-free scenarios. Although some deep learning-based methods can enhance the turbulence resistance of OAM systems in a data-driven manner, they often sacrifice transmission rates. Therefore, there is still a strong need for effective mode control technologies and robust error correction methods to provide more reliable communication performance and feasible deployment conditions for OAM/CVB-based systems at acceptable costs.

At this stage, although more efforts are required to address the challenges of transmission distance and turbulence resistance in these communication systems, OAM can still play a role in specific application scenarios. In certain situations, characterized by short distances and no or weak turbulence, such as communication between data centers or smart homes, issues related to OAM mode divergence and disturbances can be alleviated or ignored under certain conditions, thus fully leveraging the advantages of high channel density and spectral efficiency of OAM. In addition, although inter-satellite communication is not affected by terrestrial atmospheric turbulence, OAM also shows good potential for building high-speed, high-capacity communication links between satellites.

Since 2013, when terabit-scale OAM mode multiplexed fiber transmission was first demonstrated,11 OAM fiber optic communication has developed significantly over the past decade. As a special orthogonal mode basis in optical fibers, OAM modes theoretically do not offer a richer set of mode resources213 compared with other modes, e.g., the linearly polarized (LP) modes. However, their unique modal field distribution characteristics facilitate mode control for large-scale mode-division/spatial-division multiplexed fiber communication, thereby enhancing the application value. Excellent reviews of optical fibers designed for OAM transmission can be found in Refs. 213 and 214. In this context, we primarily focus on solid silica ring-core fibers (RCFs) and microstructure fibers. Solid silica ring-core fibers present a promising solution for large-capacity, low-complexity space-division multiplexed fiber communication. On the other hand, microstructure fibers offer additional dimensions for mode control, with the potential to enable novel forms of fiber transmission.

SDM techniques, which exploit the degrees of freedom in the transverse spatial domain using few-mode fibers (FMFs),215^,216 multi-core fibers (MCFs),217^–219 and few-mode multi-core fibers (FM-MCFs),220^,221 have gained significant interest over the past decades to enhance per-fiber capacity and spectral efficiency (SE). However, efforts to increase the transmission capacity of these SDM systems are often constrained by the increased complexity of multiple-input multiple-output (MIMO) digital signal processing (DSP). This complexity arises due to the involvement of more closely packed or overlapping spatial channels with significant inter-channel coupling and differential group delay (DGD).

RCFs that support the propagation of OAM modes can well cope with the above deficiencies. With the radial confinement of the ring-shaped core, the number of degenerate (or near-degenerate) modes in each higher-order mode group (MG) with topological charge |l|>0 is fixed at 4. In addition, the differential effective refractive index (Δneff) between adjacent MGs increases with |l|, thereby maintaining low inter-group coupling. These characteristics enable the expansion of multiplexed MGs with only a small-scale 4×4 MIMO equalization requiring a limited number of time-domain taps to compensate for intra-group modal crosstalk. This approach effectively reduces MIMO complexity as the mode channels within one fiber core increase. Furthermore, because the radial index of the OAM mode is limited to one in the RCF, efficient mode sorting can be achieved by considering only the azimuthal dimension. In addition, OAM modes are rotationally invariant, eliminating the need to align the modes’ rotation with phase mask patterns. These characteristics simplify the mode conversion and multiplexing process, making it easier to achieve low-crosstalk mode (de)multiplexers, particularly for large-scale FM-MCF (de)multiplexers. Leveraging these advantages, recent years have seen significant breakthroughs in high-capacity, long-distance, low-MIMO complexity OAM SDM communication systems utilizing RCFs. The following sections will provide a detailed introduction to these advancements.

The review article in Ref. 213 provides a comprehensive evaluation of short-distance, small-scale OAM multiplexed single-core RCF communication systems, with transmission distances not exceeding 20 km and no more than two multiplexed OAM MGs. In this context, we focus on large-scale OAM multiplexed single-core RCF transmission systems over longer distances, to demonstrate their excellent scalability. In 2020, Zhang et al.216 introduced a novel RCF designed to support MDM transmission using OAM modes over a single 100-km span of RCF. Figure 20(a) displays the cross-section and refractive index profile (RIP) of this RCF, respectively. The RIP features a refractive index notch atop the ring core, suppressing micro-perturbation-induced inter-MG coupling and thereby reducing fiber attenuation and mode crosstalk significantly. Notably, due to the low crosstalk of OAM modes within the RCF, the expansion of more MDM channels is feasible with just modular 4×4 MIMO processing, achieving high-capacity communication with a total transmission capacity of 2.56Tbit/s and a spectral efficiency of 256 (Tbit/s) · km. This work offers valuable insights for the development of future high-capacity fiber transmission systems [Fig. 20(b)].

Later, an impressive 1120-channel OAM-MDM-WDM transmission over a 100-km single-span RCF was showcased, employing low-complexity MIMO equalization, achieving a raw (net) capacity of 44.8 (37.3) Tbit/s and a raw (net) spectral efficiency of 22.4 (18.7) bit/(s · Hz).222 Unlike conventional RCF designs with inner and outer claddings, this double-layer RCF incorporates a high-refractive-index inner cladding—above that of the cladding yet below the core’s refractive index. This structure enables a significant Δneff greater than 1.5×10−3 at 1550.12 nm across all guided MGs, facilitating weak coupling between adjacent MGs over transmission distances up to 100 km and low attenuation.

In an endeavor to simplify the design of existing weakly coupled FMFs, refine the fiber manufacturing process, and broaden the MDM transmission bandwidth, a novel dual-layer RCF has been proposed.223 This RCF supports weakly coupled MDM transmission across the O+C+L optical wavelength bands. The fiber’s design strategy involves tuning the Δneff of specific modes by altering the refractive index of the core regions where significant transverse electric field energy is concentrated for the targeted modes, thus enhancing the Δneff between adjacent MGs. The Δneff of each MG with respect to the fundamental mode at 1550 and 1310 nm are given. One can see that the Δneff exceeds 1.5×10−3 in the C+L bands for all five guided MGs, whereas a Δneff of more than 1×10−3 is achieved among seven supported MGs in the O band. This design results in average attenuations of ∼0.21dB/km at 1550 nm and 0.39dB/km at 1310 nm. Moreover, an ultra-low power coupling coefficient of less than −32dB/km has been experimentally confirmed among all OAM MGs at 1550.12 nm.

In the transition from single-core to multi-core fibers, maintaining low MIMO complexity is crucial. Achieving low crosstalk between fiber cores, both within the fiber and at the (de)multiplexing ends, can be relatively easily accomplished by ensuring sufficient core spacing. Conventional practice suggests that when the core spacing is adequately large, the inter-core coupling becomes negligible, thereby requiring only minimal, low-complexity MIMO processing to address intra-core modal coupling.224 However, this approach results in an increased diameter of the fiber cladding, which may compromise the mechanical stability or lifespan of the fiber. Research indicates that to achieve a 1% failure probability over 20 years with a minimum bending radius of 60 mm and 100 turns, the maximum acceptable cladding diameter is 230μm.225 Thus, balancing MIMO complexity and fiber cladding diameter is essential. This can be achieved by ensuring low crosstalk both between cores and within mode groups, especially at the (de)multiplexing ends, where the complex profile of FM-MCF complicates maintaining low crosstalk. The simplified control features of OAM modes offer potential solutions, facilitating reduced MIMO complexity while keeping a relatively low fiber cladding diameter.

Li et al.226 introduced two types of multi-ring core fibers (MRCFs) designed for high-density OAM mode multiplexing. The first configuration comprises seven ring cores, with each core capable of supporting 18 OAM modes,226 whereas the second features 19 ring cores, each supporting 22 OAM modes.227^,228 The seven-ring MRCF utilizes a significant ring-core to cladding refractive index differential to confine its OAM modes, resulting in a high Δneff. This design achieves minimal inter-core and inter-MG crosstalk across the wavelength spectrum from 1520 to 1580 nm. Building on the seven-ring RCF, the 19-ring incorporates trenches to reduce inter-core crosstalk and compact the ring structure. In 2024, Huang et al. designed and fabricated a 24-km 19-core RCF, whose cross-section is shown in Fig. 20(c). This MRCF can support 266 OAM channels, demonstrating an average attenuation of ∼0.26dB/km and inter-MG crosstalk below −10dB, as shown in Fig. 20(d). Such advancements could potentially facilitate an aggregate transmission capacity in the petabit per second (Pbit/s) range and spectral efficiency of several hundred bits per second per hertz [bit/(s · Hz)] within a single MRCF. However, the two types of fibers mentioned above are still only the result of numerical simulations and no actual communication experiments have been performed.

In 2021, Zhang et al.221 were the first to present a practical demonstration of multi-core SDM/OAM-MDM per core/with WDM per mode transmission. This experiment involved a 60-km trench-assisted seven-core fiber, successfully transmitting a total of 56 OAM channels, each carrying 10 WDM channels. The Δneff between adjacent OAM MGs with a topological charge |l|≥1 exceeds 1.5×10−3. Moreover, the four modes within each mode group (±l, each carrying two orthogonal polarizations) exhibit a high degree of degeneracy. This structural arrangement aims to ensure weak coupling between high-order MGs and strong coupling within individual MGs. Subsequently, leveraging this seven-core fiber framework, Liu et al. achieved a milestone in OAM fiber-optic transmission, reaching a system capacity of 1.02Pbit/s across 24,960 channels.220 Because of the simultaneous weak coupling between fiber cores and within each core’s non-degenerate OAM MG which always contains four modes, the system achieves this by relying on fixed-size 4×4 MIMO DSP modules using no more than 25 time-domain taps per module.

In 2023, Xu et al.229 introduced a 14-km seven-core RCF capable of achieving high spectral efficiency. This fiber supports six OAM modes and two polarization modes for each OAM mode within each of the seven cores. Utilizing both MDM and WDM technologies, the system facilitates the transmission across 3360 channels. Through the application of low-complexity modular 4×4 MIMO equalization, the system demonstrates an impressive raw (net) capacity of 120.96 (100.8) Tbit/s and a significant raw (net) spectral efficiency of 241.92 (201.6) bit/(s · Hz).

Finally, we compare in detail the performance parameters of RCF in transmitting OAM modes through Table 1. These parameters include loss, crosstalk, number of modes, and transmission distance. The experimental results show that the ring-core fiber exhibits excellent performance when transmitting OAM modes, demonstrating its potential.

To overcome the inherent limitations of solid optical fibers made from quartz materials, significant attention has been directed toward researching hollow-core fibers. The concept of hollow-core fibers was first proposed by Bell Labs in 1964.230 Compared with the limitations faced by solid-core fibers, such as material absorption losses and nonlinear effects, hollow-core fibers possess superior characteristics such as dispersion modulation and minimal nonlinearity. Currently, anti-resonant hollow-core fibers, which utilize the anti-resonance mechanism to achieve localized destructive interference in the optical field, are a focal point of research.234^,235 Moreover, researchers have found that the negative curvature core structure has lower loss compared with the positive curvature core.236 These properties have made the study of hollow-core fibers very popular. In the following, we will introduce some hollow-core fibers that support the transmission of OAM modes.

In 2017, Li et al.237 explored a multimode kagome hollow-core fiber supporting terahertz OAM modes. The fiber consists of two layers of hexagonal air holes arranged in a regular pattern, and because of the circular air holes, the air holes in the center core form a negative curvature structure. This fiber makes full use of the guiding mechanism of an anti-resonant reflected optical waveguide, which is capable of supporting multiple terahertz vector modes in the central cavity, as well as terahertz OAM modes synthesized from vector modes. However, the Δneff values of nearly degenerate modes range from 10−5 to 10−4, which may result in high losses. In 2019, Tu et al.238 successfully achieved the separation of two nearly degenerate modes in each supported OAM MG through optimization of the RCF with negative curvature structure, adjusting the structural parameters of the core and cladding. This type of negative curvature RCF can guide up to eight orders of OAM modes, characterized by ultra-wide bandwidth, low scattering loss, and weak nonlinearity. Despite these developments, the loss levels in current hollow-core fibers have not yet reached those of traditional RCFs. Nonetheless, this innovative approach to mode separation presents a potential avenue for reducing losses and enhancing performance.

The common structure of a photonic crystal fiber (PCF) features a central pure silicon core surrounded symmetrically by a regular hexagonal lattice of air holes. The presence of air holes creates a significant refractive index difference between the cladding and the core, resulting in the PCF with a strong ability to confine light. The first PCF was manufactured in 1996.239 In 2012, Wong et al.240 introduced the helically twisted PCF. Light in the cladding is guided by a helical lattice of air holes and, after transmitting along a spiral path, a portion of the axial momentum is converted into angular momentum, forming OAM. However, OAM modes cannot be stably transmitted in the core of this fiber. Subsequently, Xi et al.241 detected the preservation of OAM phenomena in a helically twisted PCF with a three-bladed Y-shaped core. This design ensures the conservation of chirality for +1 order OAM modes without inter-mode coupling into the −1 order, thereby achieving distinct separation between the +1 and −1 OAM modes. Such twisted PCFs with complex core structures can be produced through the existing fiber drawing process, holding immense potential for increasing channel capacity in optical communications.

In addition to the discussed twisted PCF, another type known as ring-core PCF (RC-PCF) is used for transmitting OAM modes. The RC-PCFs generally adopt a circularly symmetric arrangement of air holes, which keeps the even- and odd-order polarized states of the vector modes in good simplicity and thus ensures the stability of the combined OAM modes. The innermost layer of air holes in the fiber is arranged in a circular pattern, facilitating the propagation of ring-shaped OAM modes, whereas the other three layers are arranged in a regular hexagon to increase light confinement to the ring core.242 This fiber effectively supports four different groups of OAM modes. At 1550 nm, the effective index difference between adjacent characteristic modes is at least 2.13×10−3. Although PCFs show tremendous potential for communication applications, due to high fiber losses and challenging manufacturing processes, researchers continue to strive for further optimization and improvement of PCFs.

Finally, due to the complexity of the RC-PCF and NC-HCF structures, significant geometrical deformation occurs during the fiber fabrication process, and mode coupling results from changes in the refractive index distribution, so there is no experimental evidence that they can support long-distance transmission of OAM modes.214 However, the multidimensional structural parameter tuning capabilities of microstructure optical fibers are not only used in the design of active optical fibers, such as PCF amplifiers to support OAM mode transmission,243 but also offer several other benefits, such as lower losses, higher nonlinear thresholds, and reduced transmission delays. These attributes make them well-suited for MIMO-free short-distance data transmissions, such as inter-computer or inter-board connections in big data centers.244 In addition, the diversity in the structures of microstructure optical fibers facilitates the development of novel optical fiber sensors based on various sensing mechanisms.245

The generation of CVBs in fiber optics and fiber lasers primarily involves stimulating and extracting suitable eigenmodes from FMFs, such as two-mode fiber (TMF) and four-mode fiber. Mode couplers play a crucial role in generating CVBs and vector beams (VBs). Long-period fiber gratings, fiber-fused mode-selective couplers, offset-spliced fibers, and tapered fibers are among the most popular types of fiber-based mode couplers. Long-period fiber gratings offer several distinct advantages, including excellent manufacturing and design flexibility, minimal reflection, and low loss. For instance, the creation of CVBs with high modal purity in fiber was successfully achieved using micro-bend long-period fiber gratings on a vortex fiber, which disperses vector modes in the LP₁₁ group.11^,246 Long-period fiber gratings are well-suited for producing CVBs when an asymmetric refractive index distribution exists across the fiber core.247 Furthermore, the controllable generation of CVBs from a long-period fiber grating, inscribed by utilizing a CO₂ laser to irradiate a two-mode fiber from one side, has recently been effectively demonstrated.248 In the context of transverse mode modulation in TMFs, the construction of CVBs external to the laser cavity has been explored. The frequency of the acoustic wave can be adjusted to meet the phase-matching requirement, and the polarization controller can be modified to transition between RPB and APB operation.249 In addition, in a previous study,250 an FMF employing a CO2 laser as the mode converter was utilized to create long-period gratings, allowing for the conversion of the fundamental mode into several higher-order modes, including OAM modes with distinct topological charges of ±1 and ±2. However, the spectrum range of high-efficiency conversion is limited to less than 20 nm. Three distinct phases of TMF long-period gratings were inscribed in a separate study.248 Notably, a 15-period grating displayed a conversion efficiency exceeding 20 dB (99%) over a bandwidth of 34.0 nm from 1529.1 to 1563.1 nm, with a peak value of 26.0 dB at ∼1533nm. Although the 15-period grating covers the majority of the C band within its 20-dB bandwidth, the lowering of the period number impacts the depth of the resonance dip, subsequently affecting the mode conversion efficiency. A dual-resonance coupling mechanism was demonstrated to enable the broadband creation of highly efficient OAM beams based on a TMF long-period grating.251 Moreover, incorporating the long-period grating into a fiber resonator enables the direct production of CVBs from fiber lasers. Chen et al.252 developed an all-fiber laser that generated CVBs using a two-mode fiber Bragg grating (TMFBG) and a long-period grating [refer to Fig. 21(a)]. In their approach, the TMFBG served as a mode selector to isolate the CVBs from hybrid modes and as a spectrum filter to adjust the laser wavelength, as shown in Figs. 21(b) and 21(c). By integrating mode-selective couplers into fiber resonators composed of SMF components, the fundamental mode was transformed into a high-order mode. Wan et al.254 utilized finite element simulation to determine the ideal diameter ratio between the SMF and TMF to satisfy phase-matching requirements, which was found to be 0.63. Furthermore, by integrating the mode-selective coupler into a Fig. 8 fiber laser, both RPB and APB could be obtained from the TMF terminal by manipulating the polarization state. They were also able to achieve femtosecond dual-wavelength soliton mode locking in a ring fiber laser with the aid of a carbon nanotube saturable absorber, confirming the broadband operating capabilities of the mode-selective coupler.255 Subsequently, they demonstrated an all-fiber CVB laser that split and excited high-order modes using a symmetric TMF coupler.256

A wavelength-division-multiplexing mode-selective coupler is proposed to efficiently convert the LP01 mode to the LP11 mode and combine high-order LP11 modes at the wavelengths of 980/1550nm,253 as shown in Fig. 21(d). The SMF was carefully matched to the TMF and pretapered to 77.5μm before being fused together during the production process using the flame brushing technique. Thanks to similar mode-selective couplers, various high-order modes, including VBs and CVBs, have been generated at a wavelength of 1.0μm in an all-fiber Yb-doped laser.257 The SMF and TMF can be offset-sliced to serve as a straightforward and efficient mode-coupling element. Grosjean et al.258 stimulated the RPB while maintaining observation of the residual fundamental mode by offset-aligning the SMF and TMF. By increasing the mismatch between the two fibers, it is possible to significantly lower the fundamental mode and increase the purity of the RPB. However, as the mismatch increased, the loss likewise increased exponentially, limiting the device’s conversion efficiency. Using the finite element analysis approach, the coupling behavior of the offset-spliced SMF and TMF is examined.259 The SMF terminal injects the light into the offset-spliced fiber, and the TMF terminal outputs it. When ΔR<5.2μm, there was a significant rise in the coupling efficiency from the basic mode to the TM01 or TE01 mode with the misfit distance growing. The basic mode and high-order modes coexisted in the TMFs after the offset-spliced fiber. By transmitting high-order modes with a TMFBG and reflecting back the fundamental mode, the mode purity may be increased. Three reflection peaks were found at 1056.0, 1054.5, and 1053.0 nm, respectively, in a typical TMFBG reflection spectrum.260 The fundamental mode was reflected back, and the necessary high-order mode was output from the TMFBG when the laser spectrum was fixed by an SMF Bragg grating.261 Sun et al. built the linear-cavity fiber laser262 and ring fiber laser263 based on the offset-spliced fiber and TMFBG to directly create RPB and APB. These cylindrical vector fiber lasers could produce continuous waves, microsecond, nanosecond, and picosecond pulses in the temporal domain. However, the significant insertion loss limited the laser’s emission efficiency and output power. For tapered fibers, the coupling efficiency of the basic mode to TM01/TE01 was 14.0%/20.6%, comparable to offset-spliced fibers.264 The insertion loss of the SMF-TMF taper is significantly lower than that of offset-spliced fibers. The CVB laser produced about 20 mW of output power while using tapered fibers, which is 1.5 to 2 times greater than when it employed offset-spliced fibers. The laser may be switched between the radially and azimuthally polarized states, precisely as offset-spliced fibers, by changing the input polarization in SMF.

The outer core radius, ring core thickness, and refractive index variations can all be utilized to identify ring-core fiber. Ramachandran et al.265 designed the pioneering ring-core fiber, whose ring core is crafted to mimic the ring shape of the CVB mode, enabling the generation and transmission of the first CVB mode. This unique fiber structure allows for the transmission of CVB modes across extended distances, reaching lengths of kilometers. In addition, a study detailing the transmission of CVBs in ring-core fibers with varying geometric structural characteristics is presented. This demonstration encompasses the multiplexed communication of coaxial high-order CVBs in ring-core fiber and the transmission of CVBs from the 2nd to 7th orders with mode purities exceeding 80%.266

Multiple SMF cores, each representing a distinct information channel, comprise the MCF. To increase fiber capacity, FMF utilizes orthogonal modes of different orders, such as CVB modes, OAM, and linearly polarized modes, as individual channels. Recently, the focus has shifted towards integrating MCF and FMF into an FM-MCF to further enhance fiber capacity. FM-MCFs capable of data rates exceeding 10 Pbit/s have been employed to showcase space-division multiplexing communication systems with over 100 spatial channels.25 However, due to the small size of an FM-MCF, coupling the beam to each fiber core poses a challenge. Various coupling techniques have been introduced for FM-MCFs recently. Low-loss, low-crosstalk devices have been developed using a free-space design approach for both 7- and 19-core multicore fibers.267 This enables the utilization of different multicore fibers with diverse structural properties across different samples with the same coupling device. Effective connection between planar silicon photonic circuits and multicore fibers is achievable through photonic wire bonding, facilitating the in-situ creation of three-dimensional interconnect waveguides that link tapered silicon-on-insulator waveguides to the fiber facet.268 Enhanced vanishing core design allows for complete return loss optimization at the pigtail and MCF interfaces, along with built-in mode adaptation from conventional SMF28 pigtails to a custom high numerical aperture MCF.269 In an FM-MCF, a 3D waveguide device featuring varying spatial path separations is fabricated using femtosecond laser direct writing.270 This 3D waveguide device connects the LP01 and LP11a modes to the FM-MCF with an insertion loss lower than 3 dB and cross talk below −36dB across waveguides. A 1-km few-mode seven-core fiber with two LP modes and two cores is utilized for four-channel multiplexing communication, showcasing bit error rate curves indicating a power penalty variation of ∼1dB due to varying waveguide bending degrees.

Developed based on two-dimensional photonic crystal technology, photonic crystal fiber (PCF) is a unique microstructure fiber with exceptional properties not typically found in conventional fibers. Modifications to the core structure of PCF are made to facilitate polarization-maintaining operation in each core and reduce mode cross-talk, thereby enhancing the stability of the generated CVBs. In a study by Zhao et al.271 in 2016, a method was introduced for creating CVBs using a specially constructed multicore PCF. However, it is important to note that solely altering the structure and characteristics of the photonic crystal fiber is insufficient to ensure single-mode transmission of the CVB mode and achieve polarization selection. Research has demonstrated that effective polarization mode selection in photonic crystal fibers can be achieved through the incorporation of metal components.272 Metal nanowires or thin films, serving as surface plasmon materials, can be integrated with photonic crystal fibers to control specific vector mode transmission within the optical fiber. Despite advancements, most devices producing CVBs are currently limited to operation within a constrained near-infrared range due to their reliance on silica photonic crystal fibers and SPR effects. To broaden the applications of CVBs within this spectral band, utilizing infrared soft glass fibers with broad transmission windows in the mid-infrared range273 presents a promising approach for achieving broadband CVB transmission. The significant nonlinearity and low transmission loss of chalcogenide glass fibers in the mid-infrared range make them well-suited for CVB supercontinuum generation.274 In the context of spatial division multiplexing, a fiber capable of supporting additional mode groups becomes necessary. The number of mode groups increases with the refractive index contrast between the fiber core and cladding, prompting interest in optimizing the design to maximize this contrast. High-order CVB multiplexing communication can be achieved through the use of air-core photonic crystal fibers.275 Within an 8.25-m-long air-core PCF, ±1st- to ±4th-order CVBs are transmitted with mode purities exceeding 76.5%. Leveraging CVB communications based on air-core PCF enables high-capacity, low optical latency short-distance optical communication.

Compared with free-space optical communication, the potential of long-distance (over 100 km) OAM/CVB mode division multiplexing communication has been demonstrated through specially designed ring-core fibers and photonic crystal fibers. However, to extend the applicability of these systems to real-world scenarios, such as underwater or terrestrial fiber optic networks, there is still a need for advanced mode field control technologies and fiber optimization solutions to reduce the complexity and costs of equipment and systems, as well as to enhance their robustness and reliability. Moreover, due to significant power loss associated with OAM/CVB modes during long-distance transmission, achieving ultra-long-distance communication comparable to existing technologies such as WDM requires not only the development of lower-loss specialty fibers and error correction techniques but also the exploration of new relay technologies to support the regeneration and amplification of signals carried by the multiplexed OAM/CVB modes. Therefore, compared with mature technologies such as WDM and PDM, OV-based communication systems have potential advantages in performance, cost, and reliability, but further efforts are needed to make them practical technologies. From this perspective, such systems are not intended to completely replace WDM and PDM technologies; rather their reliable integration with other multiplexing technologies or their differentiated supplementation in specific scenarios will remain a key focus of future works.

Furthermore, although OV-based multidimensional multiplexing technologies based on fiber transmission have demonstrated impressive performance under laboratory conditions, there is still a long way to go for the commercialization of these systems. This requires not only technical feasibility but also other conditions, such as acceptable system deployment and operational costs, as well as reliable communication performance. OAM modes are very sensitive to changes in the transmission medium, such as bending and defects in the fiber, and this sensitivity can lead to fluctuations in communication performance in real-world scenarios, making it difficult to maintain consistent service quality. Recent studies have shown that vortex waveguides developed based on vortex characteristics, screw dislocations, and higher-order topology can provide non-trivial topological protection for transmission, which may offer some promising solutions.276 However, the economic feasibility of OAM systems poses significant barriers to commercialization, as deploying OAM multiplexing requires substantial investment in new infrastructure. There is an urgent need to develop cost-effective and reliable optical fibers that support mode transmission, as well as advanced signal processing equipment and dedicated transceivers. Furthermore, affordable and robust error correction algorithms and methods are also essential to ensure the consistent performance of OAM systems under various changing environmental conditions. Ongoing research and development, along with efforts to standardize technologies, are crucial for addressing these issues and paving the way for future commercial applications.

OVs, characterized by their infinite orthogonal optical eigenmodes such as OAM and CVB modes, present a transformative opportunity for enhancing classical optical communications. OV-based communication systems benefit from the additional high-dimensional multiplexing physical degrees of freedom for signal modulation provided by the modes and maintain good compatibility with conventional multiplexing technologies such as WDM and PDM, enabling significant enhancements in channel capacity and spectral efficiency. The key components of mode (de)modulation, mode processing, and mode transmission form the foundational pillars of communication networks that leverage OVs. These elements facilitate essential functions such as signal coding, (de)multiplexing, interconnection, and transmission.

In this review, we have explored the historical development and recent advancements in optical vortex-based communications through these three critical lenses. We have systematically summarized normative definitions and research progress related to key concepts such as mode multiplexing and routing. Furthermore, we have highlighted significant challenges that remain, including issues related to scalability, integration with existing technologies, and the need for more robust error correction methods. Future research directions not only should focus on developing innovative solutions to enhance the scalability of optical vortex systems but also on exploring hybrid approaches that combine optical vortex modes with advanced DSP techniques. Addressing these challenges will not only facilitate the practical implementation of optical vortex-based communication networks but also pave the way for their application in next-generation communication systems.

Looking ahead, the future of optical communication networks based on optical vortex modes is promising. Advances in material science, photonic integration, and computational algorithms hold the potential to overcome current limitations and enhance system performance. For instance, the development of new materials with tailored optical properties can lead to more efficient mode generation and manipulation, whereas innovations in photonic integration can streamline the design and deployment of vortex-based devices, making them more accessible for practical applications. Moreover, sophisticated computational algorithms, including machine learning techniques, can optimize mode multiplexing and demultiplexing processes, improving error rates and overall system reliability. By addressing the outlined challenges and leveraging ongoing research, we aim to contribute to the practical realization of high-capacity, efficient optical communication systems. This review serves not only as a comprehensive overview but also as a call to action for researchers and practitioners in the field to collaborate and innovate, ultimately promoting the widespread adoption and application of optical vortex technologies in real-world scenarios.

Shuqing Chen is an associate professor at the Institute of Microscale Optoelectronics, Shenzhen University. His research focuses on structured light mode-division multiplexing for optical communication and computing, micro-nano photonic information processing devices, and intelligent information processing technologies. He has been recognized as one of the world’s top 2% scientists and serves as an editorial board member and young editor for journals such as Scientific Reports, PhotoniX, and Integration.

Jiafu Chen is a PhD candidate at the Institute of Microscale Optoelectronics, Shenzhen University. His research primarily focuses on high-capacity optical communications and high-performance optical computing.

Tian Xia is a postdoc at the Institute of Microscale Optoelectronics, Shenzhen University. His research interests include zone plate, beam control, holography, and metasurface.

Zhenwei Xie is an associate professor at Shenzhen University and advances optical angular momentum, developing optical vortex multiplexing mechanisms, introducing novel parity Hall effects, and predicting optical Bloch skyrmions. His research in top journals such as Nature Communications has led to 90+ SCI papers, 5600+ citations, H-index 37, and awards such as top download and ESI highly cited. He leads projects funded by the National Natural Science Foundation of China, Guangdong, Shenzhen Peacock Plan, and Huawei, and is a top 2% scientist for 2021–2023.

Zebin Huang received his BS degree from the College of Optoelectronic Engineering, Shenzhen University, China, in 2019 and his PhD from the Institute of Microscale Optoelectronics, Shenzhen University, China, in 2024. He is currently working as a postdoctoral researcher in the Department of Broadband Communication at Pengcheng Laboratory, Shenzhen, China. His research interests include optical vortice manipulations, optical diffractive neural networks, mode-division communication networks, and terahertz-integrated sensing and communication technology.

Ying Li received her PhD from Fudan University, Shanghai, China, in 2010. She was a postdoctoral fellow with the School of Information Science and Technology, Tsinghua University from June 2010 to June 2012. She is currently a professor at the College of Physics and Optoelectronic Engineering, Shenzhen University, Shenzhen, China. She is mainly engaged in research work on optical communication, microwave photonics, and structured light field regulation.

Dianyuan Fan, an academician of the Chinese Academy of Engineering, directs the International Collaborative Laboratory of 2D Materials for Optoelectronics Science and Technology at Shenzhen University. He is the director of the Technical Committee of the Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, and an executive director of the Chinese Society of Optical Engineering. He has achieved a series of internationally advanced innovative results in the fields of overall design and engineering development of laser systems, theory and application of beam transmission, and interaction between high-power lasers and matter and has won many important awards.

Xiaocong Yuan, based at Shenzhen University, directs the Nanophotonics Research Centre (NRC). He is a member of Academia Europaea and a fellow of Optica, SPIE, and the Institute of Physics. With extensive experience from Cambridge University, Nanyang Technological University, and Nankai University, he established NRC in 2013. His work focuses on light field modulation, sensing technologies, and optical interconnects, boasting over 600 SCI papers including publications in Science and Nature journals. Recognized as a highly cited scholar, he holds more than 30 patents and has received several prestigious awards. Currently, he is co-editor-in-chief of the international journal Advanced Photonics.

Biographies of the other authors are not available.