With the rapid development of the Internet and the explosion in global communication data, the fast and reliable elastic optical network (EON)^[1,2] is becoming increasingly necessary. With the change and heterogeneity of network data transmission^[3–5], the next-generation EON will become more dynamic and flexible^[6], which is primarily manifested in the dynamic adjustment of signal characteristics like transmission rate, signal power, modulation format, and forward error correction (FEC) code at the transmitter according to the needs of the transmission link and users. Since the transmitter can automatically adjust the characteristics of the signal, the receiver must be equipped with the corresponding digital signal processing (DSP) algorithms such as polarization demultiplexing, carrier phase compensation, and other algorithms to demodulate the received signal in order to maximize transmission accuracy. At the same time, the transmitted signal would be deteriorated by chromatic dispersion (CD), fiber nonlinear (NL) effect, polarization mode dispersion (PMD), polarization-dependent loss (PDL), and other transmission link impacts. Therefore, optical performance monitoring (OPM) technology^[7–9] is required in order to enable EON to be efficient and intelligent. Techniques for OPM include signal modulation format identification (MFI) and estimation of signal impairment, such as optical signal-to-noise ratio (OSNR), CD, and NL monitoring^[10,11]. MFI is a blind classification method of the transmitted signal that aims to learn the information of the modulation format from various features of different signals for later demodulation. There are many useful methods for MFI, such as signal power or amplitude distribution method^[12–15], the Stokes space method^[16–18], the peak-to-average-power ratio (PAPR) method^[19,20], and the data-aided pilot signal method^[21]. In addition to MFI, it is important for EON receivers in OSNR monitoring, since OSNR values are directly related to the bit error rate (BER) of signal demodulation and serve as a performance indicator that can reflect the quality of the transmission link. There are various methods for OSNR monitoring, including the amplitude distribution method^[22], the data-aided method^[23–25], the Stokes space method^[26,27], and the statistical method^[28–30].

Currently, the single OPM method cannot meet the needs of the fast and variable optical communication transmission networks, and the transmission quality of EON can be mastered more quickly and comprehensively by jointing multiple kinds of performance monitoring technology. With the development of neural network (NN) technology, the joint OPM technology has achieved a new milestone. Thanks to the excellent feature learning and deduction capabilities of NN technology, it is now able to recognize the minute feature differences between various signals, enhancing the speed and accuracy of performance monitoring. Mainstream NN-based OPM technologies include convolutional neural networks (CNNs)^[31–34], artificial neural networks (ANNs)^[35–39], deep neural networks (DNNs)^[40–44], and long short-term memory (LSTM)^[45,46]. Moreover, based on the structure of the NN, these techniques can be divided into multitask output and single-task cascade output. In contrast to the single-task cascade NN that requires independent network structures for different parameters, the multitask NN can use a single network structure to analyze multiple parameters, but the weights of the output layer should be set reasonably. In order to quickly and effectively perform joint MFI and OSNR monitoring on EON receivers, a concise and effective network structure should be explored.

In this paper, we proposed a multitask learning DNN based on the Stokes vector (Stokes-MTL-DNN) to perform joint MFI and OSNR monitoring, by mapping the received signal into the Stokes space and extracting the distribution of $S_1$ axis and differential phase of two polarizations as the input features of the NN. This method does not require additional hardware devices and does not sacrifice spectral efficiency. In order to validate the feasibility of the proposed method, we perform experimental tests on widely used modulation format signals in coherent optical communication, such as quadrature phase-shift keying (QPSK) and quadrature amplitude modulation (QAM). The results show that the MFI accuracy of about 28 GBaud polarization division multiplexing (PDM) QPSK/8-QAM/16-QAM/64-QAM signals can reach 100% even if the OSNR value is lower than the corresponding theoretical 20% FEC limit ($\mathrm{BER}=2.4\times {10}^{-2}$), and the mean absolute errors (MAEs) of OSNR monitoring are, respectively, 0.127, 0.244, 0.259, and 0.439 dB. Moreover, the OPM performance of this method under the influence of residual CD is also verified. When the residual CD is 800 ps/nm, the MFI accuracy of this method is still 100%, and the OSNR monitoring MAEs are, respectively, 0.168, 0.289, 0.505, and 0.875 dB. The results demonstrate that the method retains reasonable OPM performance under the influence of the fiber NL effect.

As shown in Fig. 1, at the coherent receiver, the transmitted optical signals are converted to analog electrical signals by a photodetector and then to digital signals by an analog-to-digital converter (ADC). The digital signals are then processed by using offline DSP, including downsampling, matched filtering, a CD and NL compensation algorithm, a constant modulus algorithm (CMA), and a simple phase recovery algorithm called power iteration method (PIM)^[47,48] that can extract phase noise by tracking the principal component (PC) of signals. Notice that these mentioned algorithms are modulation format-independent and may be used as general signal-processing methods. The first step of PIM for phase estimation is second-power operation of signals and covariance calculation, given as follows: $$A_n=\left[\begin{array}{ccc}\mathrm{Re}\{I{X}_n^2(1)\}& \cdots & \mathrm{Re}\{I{X}_n^2(M)\}\\ \mathrm{Im}\{I{X}_n^2(1)\}& \cdots & \mathrm{Im}\{I{X}_n^2(M)\}\end{array}\right],$$(1)$$C_n=A{}_n·A_n^T,$$(2)where $A$ is a $2\times M$ real matrix, $C$ is the covariance matrix, and $n$ denotes the $n$th block of input data. $M$ is the span of the operation, $I$ is the digital complex signal, $X$ is one of the polarization states of the signal, “Re” and “Im” denote the real operation and imaginary operation, respectively, and “T” denotes the transposition operation. Then, the first PC $\mu$ and phase noise $\phi$ of the signal can be obtained by iterative operation according to the matrix $C_n$, given as follows: $${\mu }_0=\left[\begin{array}{c}1\\ 0\end{array}\right],$$(3)$${\mu }_n=C_n·{\mu }_{n-1},$$(4)$$\overline{{\mu }_n}={\mu }_n/|{\mu }_n|,$$(5)$${\phi }_n=\{\arctan [\overline{{\mu }_n}(2)/\overline{{\mu }_n}(1)]-\pi /2\}/2,$$(6)$$\widehat{I{X}_n}=I{X}_n·\exp (-j{\phi }_n),$$(7)where $|·|$ is a modulo operation, and the original signal $I{X}_n$ is multiplied with phase noise ${\phi }_n$ after conjugate operation. Through continuous iteration, the one polarization signal after phase recovery $IX$ can be obtained. Using PIM, the phases of the signal’s two polarization states can be synchronized, making it convenient to produce the uniformly dispersed differential-phase distribution. Subsequently, the dual-polarization signals are mapped to the 3D Stokes space as follows^[18,49]: $$S=\left(\begin{array}{c}{S}_0\\ S_1\\ S_2\\ S_3\end{array}\right)=\frac{1}{2}\left(\begin{array}{c}I{X}^2+I{Y}^2\\ I{X}^2-I{Y}^2\\ 2IX·IY·\cos \mathrm{\Delta }\phi \\ 2IX·IY·\sin \mathrm{\Delta }\phi \end{array}\right),$$(8)$$\begin{gathered} \mathrm{\Delta }\phi ={\phi }_x-{\phi }_y \\ =({\phi }_{sx}+{\phi }_{xn})-({\phi }_{sy}+{\phi }_{yn}) \\ ={\phi }_{sx}-{\phi }_{sy}+{\phi }_n^{\prime }, \end{gathered}$$(9)where $S_0$, $S_1$, $S_2$, and $S_3$ are Stokes parameters that, respectively, denote the total signal power, 0° linear, 45° linear, and circular polarization components. Parameters $IX$ and $IY$ are two polarization states of the complex signals. $I^2$ denotes the square amplitude of the complex signal; for instance, the amplitudes of 16-QAM signals contain $\sqrt{2}$, $\sqrt{10}$, and $\sqrt{18}$. Parameters ${\phi }_{sx}$ and ${\phi }_{sy}$ are the modulation format phase information of two transmitted polarization signals; for instance, the phase information of QPSK contains $-3\pi /4$, $-\pi /4$, $\pi /4$, and $3\pi /4$. Parameters ${\phi }_{xn}$ and ${\phi }_{yn}$ are the phase noise of two polarization signals. ${\phi }_n^{\prime }$ is the residential phase noise caused by the nonideality of the algorithm. Since the Stokes parameters are only related to the amplitude and differential-phase $\mathrm{\Delta }\phi$ of signals, the Stokes vectors are independent of frequency offset and phase noise.

It is well known that different PDM modulation format signals express diverse spatial distributions in 3D Stokes space. There are also different distributions in 1D Stokes vector $S_1$; for instance, the 16-QAM signal has three kinds of amplitudes ($\sqrt{2}$, $\sqrt{10}$, and $\sqrt{18}$), and the cluster levels of $S_1$ in Eq. (8) are $-8$, $-4$, 0, 4, and 8, but the cluster level of the QPSK signal (amplitude is $\sqrt{2}$) is only 0. Therefore, vector $S_1$ can be utilized as the feature to describe different modulation formats. Simultaneously, we discovered that the $\mathrm{\Delta }\phi$ between two polarization signals also has different distribution features; for example, the $\mathrm{\Delta }\phi$ of PDM QPSK signals in Eq. (9) are $-\pi$, $-\pi /2$, 0, $\pi /2$, and $\pi$. The cumulative distribution function (CDF) is used to reflect the distributions of $\mathrm{\Delta }\phi$ intuitively and simply. Then the processed samples are projected into the axis with 80 bins and 100 bins for $\mathrm{\Delta }\phi$ to obtain the $S_1$ distribution features. Figures 2(a) and 2(b) show the $S_1$ and $\mathrm{\Delta }\phi$ distribution features of PDM QPSK/8-QAM/16-QAM/64-QAM signals under low and high OSNR values, respectively. It can be seen that the $S_1$ distribution of QPSK when OSNR is 9 dB is similar to that of 64-QAM when OSNR is 30 dB, but their $\mathrm{\Delta }\phi$ distributions are significantly different. As a result, taking $S_1$ and $\mathrm{\Delta }\phi$ as features can increase the discrimination of different signals and improve the accuracy of OPM by using DNN. Then, the signals are trained and tested by Stokes-MTL-DNN to obtain the corresponding modulation format information, which is applied by the subsequent DSP algorithms to finally demodulate data. In order to facilitate the training of the NN, the modulation format needs to be one-hot-encoded first, such as [0,0,0,1], [0,0,1,0], [0,1,0,0], [1,0,0,0] corresponding to QPSK, 8-QAM, 16-QAM, and 64-QAM, respectively. The structure of the proposed Stokes-MTL-DNN is shown in Fig. 1, the distribution features of $S_1$ and $\mathrm{\Delta }\phi$ are input into the NN for dimension reduction, respectively, and then the two features are concatenated to extract the features by the four hidden layers, where the activation function is LeakyReLU and the neurons are 128, 64, 32, and 8, respectively. Finally, the modulation format and OSNR value are output simultaneously at the output layer, where the neurons of MFI are 4 and OSNR is 1. Among them, MFI is considered a classification task with softmax and categorical cross entropy as the activation function and loss function, respectively; OSNR monitoring is considered a regression task with linear and logcosh as the activation function and loss function, respectively. The Stokes-MTL-DNN is implemented with the Keras deep-learning framework.

A series of tests employing flexible format transmitters is carried out to verify the feasibility of the proposed approach, as shown in Fig. 3. At the transmitter, a word length of $2^{15}-1$ pseudo-random bit sequence (PRBS) is generated and mapped into QPSK or mQAM, which is sampled twice per symbol. An integrated ${\mathrm{LiNbO}}_3$ I/Q modulator operated by two ports of digital-to-analog converter (DAC) runs at 64-GSa/s with 25-GHz analog bandwidth to generate 28 GBaud QPSK/8-QAM/16-QAM/64-QAM optical signals with an external cavity laser (ECL), in which the center wavelength is $\sim 1549.32\mathrm{nm}$ and the linewidth is $\sim 100\mathrm{kHz}$. Notice that the light generated by the ECL passes through a 90:10 fiber-optic coupler, where 90% of the light is used as signal carrier and 10% as a local oscillator (LO). Then, these signals are operated by a coupler, two polarization controllers (PCs), two variable optical attenuators (VOAs), an optical delay line (DL), and a polarization beam splitter (PBS) to generate PDM signals. Different amplified spontaneous emission (ASE) noises are generated by adjusting the amplification factors of the erbium-doped fiber amplifier (EDFA) to alter the OSNR values of the input optical signal, including 9–16 dB for QPSK, 13–23 dB for 8-QAM, 15–30 dB for 16-QAM, 18–30 dB for 64-QAM, and the actual OSNR value is monitored by an optical spectrum analyzer (OSA). At the receiver, an optical bandpass filter (OBPF) with 0.8 nm bandwidth is utilized to filter out the outband noise to improve the performance. The signals after OBPF are merged with the LO at the polarization diversity hybrid and detected by balanced photodetector (BPD) as a four-way electrical signal for offline DSP and OPM. In this scheme, we generate 280 data sets for each OSNR value of different modulation formats, including 2240 data sets for QPSK, 3080 data sets for 8-QAM, 4480 data sets for 16-QAM, 3640 data sets for 64-QAM, and each data set that indicates $S_1$ or $\mathrm{\Delta }\phi$ distribution feature is generated through 20,000 symbols. Meanwhile, we randomly select 80% of the data sets as training data sets, and the remaining are testing data sets.

The experimental data are processed according to the principle of Fig. 1, the number of taps in the CMA algorithm is 25, the histogram distribution ranges of $S_1$ and $\mathrm{\Delta }\phi$ are $(-1.8,1.8)$ with 80 bins and $(-\pi ,\pi )$ with 100 bins, respectively. In order to filter out distribution noise, smooth filtering is performed on the $S_1$ feature, and the size of the sliding window is 5. Subsequently, the distribution features of $S_1$ and $\mathrm{\Delta }\phi$ are normalized and then put into the Stokes-MTL-DNN, where the first hidden layer is L2 regularized with weight 0.1 and followed by a dropout layer with weight 0.1 to improve the generalization ability of the network. The epoch is set at 500, using MAE to evaluate OSNR monitoring. The MFI and OSNR monitoring results of back-to-back (BTB) PDM QPSK/8-QAM/16-QAM/64-QAM are shown in Figs. 4 and 5. Identification accuracies of 100% for four modulation formats are achieved even when the OSNR values are lower than the corresponding theoretical 20% FEC limit, such as 9 dB for QPSK, 13.9 dB for 8-QAM, 15.8 dB for 16-QAM, and 21.5 dB for 64-QAM. Moreover, the MAEs of OSNR monitoring for BTB PDM QPSK/8-QAM/16-QAM/64-QAM are 0.127, 0.244, 0.259, and 0.439 dB, respectively, and the MAE of each OSNR monitoring with different modulation formats is shown in Fig. 5. It can be seen that all OSNR monitoring values are within a reasonable range, which proves the feasibility of the proposed scheme in MFI and OSNR monitoring.

In order to further investigate the effect of residual CD on the proposed method, we add different CD values in all experimental data sets by using a DSP algorithm, such as 0 ps/nm, 400 ps/nm, and 800 ps/nm. As shown in Fig. 6, the MAE of OSNR generally shows an upward trend with the accumulation of residual CD values, and the high-order modulation format is sensitive to CD due to its large number of energy levels, which leads to changes in the distribution features and becomes less obvious, but the proposed method still has a certain CD tolerance. Meanwhile, the identification accuracies of all modulation formats are still 100% under the different residual CD and OSNR values, as shown in Table 1.

As we all know, fiber NL impairment will occur in long-distance transmission or under high launching optical power, which can decrease the performance of signal demodulation. In order to verify the NL impairment tolerance of the proposed method, we used the commercial software VPI Transmission Makers to carry out the numerical simulation, as shown in Fig. 7. The span length of the standard single-mode fiber (SSMF) is set at 80 km with a dispersion of 16.8 ps/nm, an attenuation coefficient of 0.2 dB/km, and an NL coefficient of $1.27{\mathrm{km}}^{-1}·{\mathrm{W}}^{-1}$. 28 GBaud PDM QPSK/8-QAM/16-QAM/64-QAM signals with different OSNR values are generated and then transmitted through a long-haul transmission link, such as 4000 km SSMF for QPSK with OSNR 9–16 dB, and 8-QAM with OSNR 13–23 dB, 2000 km SSMF for 16-QAM with OSNR 15–30 dB, and 1040 km SSMF for 64-QAM with OSNR 18–30 dB. In order to investigate the NL impairment caused by high launching power (LP), the LP range is set from $-1\mathrm{dBm}$ to 5 dBm. For the signals obtained from the above simulations, only the dispersion effect generated by long-distance transmission is compensated; the rest of the DSP processing flow is shown in Fig. 1, thus obtaining the signals with different NL impairments. Subsequently, the proposed network is trained by using only transmission signals with 1 dBm LP, and the joint MFI and OSNR monitoring performance for signals with different LPs are tested. The MAEs of OSNR monitoring for all modulation formats with varied LPs are shown in Fig. 8; it can be seen that when the LP is less than 1 dBm, there is no strong NL effect due to the low LP, and the signal is mostly affected by additive Gaussian white noise. As the LP increases, the NL damage within the fiber gradually increases the impact on the optical signal, resulting in the distribution features of $S_1$ and $\mathrm{\Delta }\phi$ becoming less obvious, and thus the MAE increases. The MAE of OSNR monitoring with different LPs is 1.27, 0.68, 0.26, 1.05, 1.99, 2.66, and 3.56 dB, respectively, and the identification accuracies for all modulation formats with different LPs are still 100%. These results show that the proposed method has high tolerance to the fiber NL impairment and can be applied in long-haul transmission.

We compare the BTB experiment system performance of the proposed method and a similar method using the processed Stokes vector of signals^[48]. The performance of MFI and OSNR monitoring of these methods is shown in Table 2. Notice that both approaches employ 28 GBaud experiment data rates, while the processed Stokes vector method takes a single input but combines the Stokes vector $S_2$ and $S_3$ as one feature. As shown in Table 2, the proposed method can realize more modulation formats classification and lower MAE of OSNR monitoring as well as lower complexity, and we believe that this method can suit the demands of the rapid and intelligent EON in the future.

In this paper, we proposed a convenient and intelligent method based on DNN for jointing MFI and OSNR monitoring. The feasibility of the method in PDM QPSK/8-QAM/16-QAM/64-QAM signals is validated by BTB transmission experiments, and the tolerance to residual CD is investigated. Finally, we designed a long-distance transmission simulation link to investigate the tolerance of the proposed method to the fiber NL impairment. The results demonstrate that even if the signal OSNR value is less than the 20% FEC threshold, all modulation formats have 100% identification accuracy, and the MAE of OSNR monitoring remains at a reasonable level. With the rapid development of fiber optic communications and the proposal of 6G technologies^[50,51], the role of OPM technology will become more important. The proposed method does not add extra hardware in the link; moreover it can perform identification and detection with high accuracy and speed, and we believe that this method can be applied to the next-generation EON.