As an optimal communication method for high transmission rate, large capacity, and low power consumption of the satellite system, laser communication technology breaks through the limitation of microwave application in satellite communication^[1]. In ground-to-satellite/satellite-to-ground links, the laser beam is in direct contact with the atmosphere, and the channel state is not stable due to spatial and temporal changes in the refractive index of the atmosphere, which poses a serious challenge to the performance of optical communication systems^[2,3]. Adaptive modulation and coding (AM&C) technology can freely select modulation and coding modes according to the specific channel state; some researchers have applied AM&C technology to the next generation of global navigation satellite systems, which proves that the AM&C scheme can effectively improve system throughput^[4].

In order to ensure the successful transmission of information, the receiver must determine the modulation mode of the transmitter; if the classification of the modulation style is incorrect, the whole transmission may fail due to the demodulator demodulating the information wrong. Therefore, the automatic modulation classification (AMC) scheme is proposed^[5]. AMC algorithms fall into two categories: the likelihood-based classifier^[6–8] and the feature-based classifier^[9,10]. The main disadvantage of the former is that it is computationally expensive and sensitive to damage such as phase and frequency offset, while the feature-based method is robust and less complex to implement than the former. Researchers proposed schemes for AMC from various perspectives, such as the combination of constellation diagram and depth learning and dictionary learning-based AMC framework^[11–13], that have played an important role in many fields such as wireless communication^[14,15].

In the 1990s, it was discovered that signal characteristics could be extracted from the constellation of modulated signals^[16], but the difficulty is to extract effective features, and the modulation format continues to evolve^[17–19]. Fractal features have excellent performance in this respect. In 1975, Mandelbrot put forward the concept of fractals for the first time, to the best of our knowledge, which describe the shape that is similar to the whole in some way, and which has self-similar properties in structure, function, information, time, etc.^[20]. Subsequently, the fractal theory gradually developed, and the fractal methodology, which was born from it, has played a great value and scientific methodological significance in various fields, including the application of feature extraction^[21–23].

Although many methods have been proposed to realize AMC, no one has attempted to study automatic modulation recognition from the perspective of fractal and machine learning. In this Letter, a modulation classification method in combination with partition-fractal and support-vector machine (SVM) learning methods is proposed to realize no prior recognition of the modulation mode in satellite laser communication. The effectiveness and accuracy of this method are verified under nine modulation modes. The simulation results show when the signal-to-noise ratio (SNR) of the modulated signal is more than 8 dB, the classifier accuracy based on the proposed method can achieve more than 98%, especially when in binary phase shift keying (BPSK) and quadrature amplitude shift keying (QASK) modes, and the classifier achieves 100% identification whatever the SNR changes to. The research has certain theoretical significance and application value for realizing AMC in the satellite communication field.

Different signal modulation modes have different signal characteristics, in which constellation diagrams can reflect amplitude and phase information. In order to realize AMC in satellite communication systems and make correct decisions on the received signal modulation mode, it is necessary to consider how to make use of the features represented by constellations. To solve this problem, we propose the following scheme: the whole process of the judgment work is as shown in Fig. 1. From the figure, the ground station modulates the data in a modulation mode and transmits the data to the satellite terminal, and the satellite terminal recognizes and classifies the received signals according to the proposed method, determines the demodulation mode, and then obtains the correct data, thus completing the one-way communication process.

The proposed method consists of two parts: the partition-fractal and SVM-based multi-modulation pattern classification methods.

The first step of the method is to obtain the constellation features. As the object of feature extraction, the constellation diagram is composed of a number of modulated signal vector endpoints, which have self-similar properties in whole and part, but the distribution of endpoints is different in different modulation modes, so the fractal method is suitable for fractal feature extraction of the constellation diagram. In this part, a partition-fractal method is proposed to extract the fractal box dimensions of the modulated signal constellation diagram, as shown in Fig. 2.

In the figure, the key operational steps are as follows.System.Xml.XmlElementSystem.Xml.XmlElement

The fractal dimension D can be described as^[24]where for a bounded set A, it is self-similar when A is the union of the $N_r$ distinct (no overlapping) copies of itself, each of which is similar to A scaled down by a ratio $r$.

In this step, the DBC method is applied to find $N_r$ and calculate the D of the segmented constellations, where, the minimum and maximum gray levels of the image in the $(i,j)$ grid fall in box numbers $k$ and $l$.

The contribution of $N_r$ in the $(i,j)$ grid is described as

Taking contributions from all grids,

To better characterize the constellation diagram features, the fractal feature matrix and gray feature matrix form the final feature matrix, and the calculation process is shown in Fig. 3. Firstly, based on the partition-fractal method, the calculated box dimension values are arranged in a sequence from left to right and from top to bottom, and thus a box dimension matrix of $1\times 64$ (B1, B2, …, B64) can be obtained, which is the fractal characteristic matrix of the constellation diagram. Secondly, to exclude the influence of the region range, four gray level co-occurrence matrices with different directions (0, 45, 90, 135 deg) are selected, then four kinds of gray level co-occurrence matrix statistics are calculated and normalized including contrast, inverse distance, entropy, and autocorrelation, and the average value and variance are taken as the final extracted features.

Thus, the constellation feature set can be obtained, and the final feature matrix contains the geometric scale information of the constellation and comprehensive information about the direction, adjacent interval, and change amplitude, which lays a foundation for later classification.

The second step of the method is to train the feature data set. The SVM is an effective method to implement the classifier and has excellent performance in preventing overfitting^[25]. To find the optimal hyperplane from the selected modulation modes to complete the multi-modulation mode classification, the SVM method is applied to achieve the learning and classification based on the constellation diagram feature matrix.

Consider the high requirement of recognition accuracy in satellite communication systems, where the accuracy of the classifier is chosen as the performance evaluation standard, and the formula is shown in Eq. (4), where $n_c$ is the number of constellations correctly classified, and N is the total number of test constellation samples:

According to the above description of the proposed method, the detailed steps for constructing a classifier and evaluating its performance based on the proposed method are as follows.System.Xml.XmlElementSystem.Xml.XmlElementSystem.Xml.XmlElementSystem.Xml.XmlElementSystem.Xml.XmlElement

Firstly, we analyzed the performance of the classifier constructed according to the proposed method under different SNR. At each SNR, 100 constellation diagrams are obtained with a pixel size of $512\times 512$ in a single modulation mode, and there are nine types of modes. Then, get constellation sets of different modulation modes under different SNRs, and divide them into training constellations and test constellations according to the ratio of 7:3. Finally, we construct the classifier and verify its performance; the simulation parameters are shown in Table 1, and the accuracy results of classifier with SNR changing are represented in Table 2.

It is apparent from Table 2 that as the SNR increases the accuracy of the classifier gradually increases and finally becomes one, which means 100% recognition can be achieved. The results prove that the classifier constructed by the proposed method can realize classification, and the classification accuracy is higher when the SNR is large.

To verify the superiority of the proposed method based on partition-fractal and SVM learning methods, we compare the classifier performance with other learning algorithms, including the SVM learning algorithm, bagging ensemble learning algorithm, $k$-nearest neighbor (KNN) search learning algorithm, classification tree learning algorithm, and AdaBoost ensemble learning algorithm. The simulation parameters are the same as above, but the difference lies in different learning algorithms replacing SVM of the proposed method; the simulation results are as shown in Fig. 4.

From the graph above, the accuracy curves of the five learning algorithms all show an upward trend with the increase of the SNR, and the performance of the classifier based on different learning methods are as follows. The AdaBoost method is the one with the worst performance, bagging, KNN, and the classification tree are a little different in performance under different SNRs and are better than AdaBoost, and the best-performing method is SVM learning. The classifier based on SVM still has more than 90% accuracy even under low SNR and achieves more than 98% accuracy when the SNR is more than 8 dB, which is obviously superior to other algorithms. Thus, this result verifies the superiority of the proposed partition-fractal method and SVM learning method in realizing AMC. The reason for such curve results may be that SVM has good performance in preventing overfitting, so the constructed classifier also performs well in the test set, and AdaBoost has an overfitting phenomenon.

The results of further analysis of classification accuracy in different modulation modes are shown in Fig. 5. It is apparent from the figure that the classifier based on the SVM learning method has good recognition performance in different modulation formats compared with other learning algorithms. In particular, for BPSK and QASK modes, the classifiers are 100% identifiable regardless of the SNR change, while the performance is slightly poorer in the binary amplitude shift keying (BASK) mode. As a whole, the classification accuracy of SVM-based classifiers in phase shift keying (PSK) and frequency shift keying (FSK) patterns is higher than that of amplitude shift keying (ASK), which indicates that SVM-based classifiers are better at classifying PSK and FSK patterns.

In summary, to realize AMC of satellite laser communication in ground-to-satellite/satellite-to-ground links, a modulation classification method in combination with the partition-fractal method and SVM learning method is proposed that extracts and trains constellation diagram features. From simulation results, application of the proposed methodology using the partition-fractal method and SVM learning for no prior recognition of the modulation mode is feasible, and the classifier can achieve more than 98% accuracy when the SNR is more than 8 dB, especially when in BPSK and QASK modes, where the classifier achieves 100% identification whatever the SNR changes to.

Therefore, by adopting the proposed method, we achieve AMC. This method does not limit the type of signal modulation for strong scalability and continues to learn in the case of an increased modulation mode to achieve more modulation mode identification. Future research is considered to be carried out from two aspects: the first is to realize the modulation format identification of multi-carrier modulation signals in combination with feature extraction and machine learning to solve the identification problem in this field; the second is to realize the modulation format identification from the perspective of deep learning, such as convolutional neural network (CNN). The research has certain theoretical significance and application value for realizing AMC in satellite communication fields.