Distributed acoustic or vibration sensors (DASs/DVSs) have been widely used in perimeter security^[1], pipeline safety alarm^[2,3], structure-health monitoring^[4], and traffic control^[5], due to their advantages, including simple structure, accurate positioning ability, and ultralong sensing range for disturbance detection. To date, only the disturbance location no longer meets the requirements under complex and various scenarios, where the identification of disturbance signals should be resolved for future investigation or application^[6]. Utilizing the backscattered signals and coherent detection, researchers can simultaneously obtain the time, space, and frequency feature of disturbance signals imposed on optical fibers^[7–11] via the phase-sensitive time-domain reflectometer ($\phi \text{-}\mathrm{OTDR}$) scheme, which is of great value for pattern recognition.

$\phi \text{-}\mathrm{OTDR}$ was first proposed by Taylor et al. in 1993^[12]. Recently, numerous pattern recognition methods based on deep learning have been applied to $\phi \text{-}\mathrm{OTDR}$. Common algorithm models include vector machine (SVM), long short-term memory (LSTM), artificial neural networks^[13,14], and convolutional neural networks (CNNs)^[15]. Among them, CNN is the most representative of deep learning, which is currently mainly used in image recognition and speech recognition. For example, inverted residual CNN (IRCNN) uses inverted residual blocks to save feature information, which greatly improves the accuracy of feature classification ^[16]. Its accuracy far exceeds other types of neural networks. In 2017, Aktas et al.^[17] used a short-time Fourier transform (STFT) to extract the time-frequency information of the signal with CNN to identify and classify walking, digging, strong wind, and equipment noise in $\phi \text{-}\mathrm{OTDR}$. The accuracy is over 93%. In 2018, Xu et al.^[18] used STFT to obtain the time-frequency characteristics of intrusion signals, converted different types of vibration signals into spectrograms, and realized the identification and classification of four types of intrusion events. Zhao et al.^[19] proposed a new method based on the Markov transition field (MTF) and deep learning to classify vibration events and measure vibration frequencies. In addition, some other methods, such as wavelet decomposition^[20], wavelet energy spectrum analysis^[21], and empirical mode decomposition^[22] have also been used to extract time-frequency domain features for pattern recognition.

In addition, researchers pay attention not only to time-frequency features but also to temporal-spatial features of signals. Wang et al.^[23] proposed a method based on the deep dual-path network and a time spectrum for $\phi \text{-}\mathrm{OTDR}$ event recognition by establishing the time spectrum of the disturbance signal to attain the temporal-spatial features and achieved 94.3% accuracy with seven types of disturbances. In 2021, Shi et al.^[24] proposed a deep-learning-based multiradial distance event classification method. The method can distinguish both event types and radial distances by extracting the spatiotemporal data matrix. The aforementioned works only focus on the time-frequency features or the space-time features. The two types of features have not been considered together for comparative experiments.

In this paper, the image processing and STFT technology are used to extract disturbance information in the time domain, the frequency domain, and the space domain. We use a CNN to train and classify the data. The experiment results show that the temporal-spatial image can achieve 99.56% classification accuracy on three kinds of events with 3600 sample data sets, whereas the time-frequency image achieves 98.23%. Since the extraction of temporal-spatial features does not require accurate location, the signals can be directly converted into images, and CNN can directly extract temporal-spatial features. However, when extracting time-frequency features through STFT, it is necessary to identify the exact location of the vibration event, leading to some errors in the positioning process. In this study, the results of pattern recognition of temporal-spatial and time-frequency features are compared and analyzed from several aspects, which provides reference value for the feature extraction method in $\phi \text{-}\mathrm{OTDR}$ pattern recognition.

The rest of this paper is structured as follows. Section 2 introduces the process of data acquisition and processing, including experimental settings, data processing methods, data set composition, and pattern recognition methods. In Section 3, two methods of pattern recognition are compared and analyzed by experiments. Thus, it is proved that the classification effect of temporal-spatial images is better than that of time-frequency images. Finally, Section 4 summarizes the paper.

The setup of the involved $\phi \text{-}\mathrm{OTDR}$ system is shown in Fig. 1. Figure 2(a) shows the developed prototype $\phi \text{-}\mathrm{OTDR}$ instrument, and Fig. 2(b) shows the deployed fiber in experiment. An ultranarrow linewidth laser is used as the light source. The output light of the laser is divided into two beams by a 99:1 coupler. The 99% light serves as the probe light, and the 1% section is the local oscillator light. The probe light is modulated into pulsed light by an acoustic-optic modulator (AOM), amplified by an erbium-doped fiber amplifier (EDFA). Then the probe pulse light is injected into the sensing fiber through the circulator. Then, the Rayleigh backscattered signal and local oscillator light are mixed in a 50:50 coupler and converted into an electrical signal by a balanced photodetector (BPD). The electrical signal is received by the data acquisition card (DAQ). Corresponding digital signal processing (DSP) is performed on the collected digital signals to obtain information, including the location of the disturbance and the frequency of the disturbance signal, which is shown in Fig. 2. The probe pulse width is set as 100 ns, and the repetition period is 2 µs. The frequency shift of the AOM is 80 MHz, and the DAQ sampling rate is 200 MS/s. The spatial resolution of the $\phi \text{-}\mathrm{OTDR}$ system is 10 m.

When the vibration signal is applied to the fiber, the phase of the Rayleigh backscattering (RBS) signal in the disturbed region is modulated accordingly. The RBS signal after the disturbance can be expressed as $$I_{\mathrm{com}}(t)=E_s{E}_{\mathrm{LO}}\cos (\mathrm{\Delta }\phi )+j{E}_s{E}_{\mathrm{LO}}\sin (\mathrm{\Delta }\phi ),$$(1)where $E_s{E}_{\mathrm{LO}}$is the intensity of the RBS signal, and $\mathrm{\Delta }\phi$ is the phase of the RBS signal. $I_{\mathrm{com}}(t)$ is converted into a trace matrix. The received RBS signal can be converted to the data matrix, as shown in Fig. 3. The first dimension of the matrix is named “fast time,” which represents the distance along the fiber. The second dimension of the trace matrix, which is termed “slow time,” is the time axis of the RBS signal at each location of the fiber^[25]. The angle function is used to extract the phase of the trace matrix signal, as shown in the following: $$\mathrm{\Delta }\phi =\text{angle}(E_s{E}_{\mathrm{LO}}{e}^{j\mathrm{\Delta }\phi }).$$(2)

Then STFT is performed on the obtained disturbed signal to obtain the time-frequency diagram^[18]. The basic calculation is $$S(z,f)={\int }_{-\infty }^{\infty }i(z)g(z-\tau )\exp (-j2\pi fz)\mathrm{d}z,$$(3)where $i(z)$ is the disturbance signal; $g(z-\tau )$ is the time window centered on $\tau$; and $\exp (-j2\pi fz)$ is a modulation operator that converts the signal from the time domain to the frequency domain.

After selecting the STFT window function, the resolution is fixed when the window length function is determined. Since the time resolution is inversely proportional to the frequency resolution, they cannot be optimized simultaneously. Therefore, choosing the appropriate window function type and reasonable window length is vital. Here we use the Hanning window function to intercept the signal to guarantee fine-frequency resolution and less spectrum leakage. Its expression can be written as $$w(n)=\frac{1}{2}\{1-\cos \left[\frac{2\pi (n-1)}{N}\right]\},0\le n\le N-1,$$(4)where $N$ is the length of the window. To obtain frequency resolution and facilitate calculation, the window length is set to 8192.

The data set greatly influences the recognition performance, and a reasonable data set structure is crucial for pattern recognition. Common vibration signals include mechanical activity, walking, wind blowing, rain, knocking, shaking, etc. In this paper, three typical event types, i.e., background noise, knocking, and wind blowing, are selected. The knock signal is obtained by striking with a hammer. A fan generates the wind blowing signal to simulate the natural wind blowing in the virtual environment. Background noise is obtained when the ambient environment is relatively quiet without any vibration event. The composition of the data set is shown in Table 1. The time-domain waveforms of different vibration events are presented in Fig. 4.

We have selected a range of 10 m in the spatial neighborhood of the vibration signal. Each temporal-spatial image represents a spatial length of 10 m and a temporal length of 1 s, shown in Figs. 5(a)–5(c), where the unit on the color bars is the radian (the unit of phase). The vertical and horizontal direction of the temporal-spatial image represents position and time, respectively. The time duration of time-frequency images is 1 s, which is shown in Figs. 5(d)–5(f). For the time-frequency image, the horizontal axis presents time too, whereas the vertical axis presents frequency. The unit of the color bars in Figs. 5(d)–5(f) is the decibel, where 0 dB refers to the maximum value of the whole spectrum along the position.

Deep-learning methods such as CNN can automatically learn the features of data sets. ResNet50 has a deep network level and few parameters, achieving good results in image data sets in other fields. Consequently, it is applied to $\phi \text{-}\mathrm{OTDR}$ event classification. ResNet50 increases the network depth by stacking the remaining blocks to solve the network degradation problem and the vanishing gradient problem^[26–28]. Here we adopt ResNet50 network as the recognition classifier, as shown in Fig. 6. ResNet50 contains 50 conv2d operations, which are divided into five stages. The first convolution layer has 64 kernels. The size and maximum pooled stride of each kernel are $7\times 7$ and 2, respectively. The last four stages have 9, 12, 18, and 9 convolution layers, also known as residual blocks, as shown in Fig. 7. x is the input to the residual block. F(x) is the output after linear change and activation. The figure shows that in the residual network, F(x) adds the input value x of this layer before activation after the linear change of the second layer, and then carries out the output after activation. X is added before the second layer output value is activated. This path is called a shortcut connection. The residual block is used to deepen the network. With the increase of the network, the global feature vector of the feature image is extracted (2048, 7, 7). The average pooling layer converts it into an eigenvector. After passing through the dropout layer, the 2048-dimensional feature vector is connected to the full connection layer. Finally, the classifier calculates the feature vector and outputs the category probability. For training, the model performs L2 regularization optimization on epochs. In addition, we trained the model using feature images to verify the effect of time and space on classification accuracy. The categorical cross-entropy loss uses the Adam optimizer and the ReLU activation function to speed up the computation. The model input is a $1\times 3\times 224\times 224$ feature vector, and then the data are processed through two-dimensional convolution, max pooling, and fully connected layers.

Here, the ResNet50 classifier is used to train and validate the temporal-spatial and time-frequency images, respectively, and the cross-validation method is used to evaluate the classification results. The learning rate is set as 0.003. In deep learning, the principle of small-batch data processing is generally adopted. The batch size is 25, and the training epoch is 100. We collected 1200 sets of three signals: knock around the fiber with a hammer, the wind blowing optical fiber, and background noise. The ratio of the training set and validation set is 7:3. The loss and accuracy are obtained with each iteration, and the hyperparameters are changed to optimize the next training. Each network is tested in the test phase every 100 iterations. The resultant training loss and validation accuracy curves are shown in Fig. 8.

During the training on time-frequency images, the training loss, representing the network error, decreases slowly after 50 epochs until it hovers around 0.03, and the best test accuracy is 98%. In contrast, the test accuracy of the temporal-spatial image is 99%, and the training loss reaches 0.01 after 30 epochs. The results show that the classification performance based on the temporal-spatial images is better. The classification results are shown in Table 2, which shows that classification accuracy based on the temporal-spatial images achieves 99.49%, whereas the classification accuracy based on time-frequency images is 98.23%. The accuracy of temporal-spatial images is 1.33% higher than that of the time-frequency images. From the average training time per step, the temporal-spatial map as input is improved by 3 s compared to the time-frequency map as input. However, from Table 3 it is clear that the recall and f1-score of the input time-frequency image reach 100%, indicating that the model has a better recognition effect on the time-frequency characteristics of continuous signals. The confusion matrices are shown in Fig. 9. The knocking signal is most easily confused compared with the other two types of signals, primarily based on time-frequency characteristics. A possible explanation is that the knock signal is a single point tapping signal that is transient in the time domain. There is a time difference between the two disturbance signals. The signal feature image will contain the interference information of the external environment.

The experimental results show that the classification effect based on the temporal-spatial image is better than that of the time-frequency image. In preprocessing the two signals, the extraction of the temporal-spatial feature does not require precise positioning in advance. Taking the distance within the range of 20 m around the perturbation position, we have converted the signal of this distance into a temporal-spatial image. The temporal and spatial correlation of the signal is preserved, and the intensity information is extracted from the image. The temporal-spatial features are directly extracted by the CNN model, which avoids manual processing during the classification, thereby reducing the workload and retaining the features of the original signal. However, the time-frequency feature extracted by STFT must locate the disturbance signal. The time-frequency features extracted by STFT need to locate the disturbance signals and extract the signal features. In practical applications, most of the collected signals are discontinuous signals. Due to external disturbance, it is difficult to accurately extract target disturbance information, and there may be some information loss. Therefore, the extraction of time-frequency feature information is greatly affected by the environment, resulting in a slightly worse classification effect.

This paper compares and analyzes the disturbance events classification methods in $\phi \text{-}\mathrm{OTDR}$ using different features of CNN, namely, temporal-spatial features and time-frequency features. Through the experiments on a 3600-sample data set, temporal-spatial and time-frequency images are directly used as the input of the ResNet50. The temporal-spatial images based classification accuracy achieves 99.49%, and the classification accuracy of time-frequency images is 98.23%. Training accuracy, valid accuracy, average training time, loss curve, and confusion matrix are used as evaluation criteria. Due to the disturbance of the external environment, the space range of time-frequency feature extraction is short, and there is interference in feature extraction for the transient signals, such as knocking. The temporal-spatial image of the signal has good space–time correlation, and the extracted features contain complete information. Therefore, the classification effect of temporal-spatial images is better than that of time-frequency images. The recognition results of the two feature images provide meaningful reference for the further study of pattern recognition in $\phi \text{-}\mathrm{OTDR}$.