The sensing rail section is from South Shawo Bridge in urban area, to Dujiakan Bridge on the southwest of Beijing, passes 6 ballasted track bridges (inset I), turns into the ballastless track (inset II) then and goes over Beijing Garden Expo on simple beam bridges (inset III). Out of the sensing rail line, two high-speed rail tracks converge on Dujiakan Bridge. The laser interferometer system is deployed near Dujiakan Bridge. Light from laser source is divided by OC into two beams: one is the reference beam, the other is sensing beam, modulated by AOM, then sent into fiber cable along the railway and looped back at South Shawo Bridge. These two beams interfere at PD. The output signal is taken by DAQ and analyzed by FPGA module. The extracted phase changing signal is put into neural network Res-LstmNet for train localizing. Beijing-W Beijing west railway station. OC optical coupler, AOM acousto-optic modulator, PD photodiode, DAQ data acquisition system, FPGA field programmable gate array, ResNet Residual Neural Network, Lstm Long short-term memory.

Results

a The time-space waterfall plot of 3 high-speed trains, which is the result of Res-LstmNet. Based on the vibrations detected by laser interferometer, location labels corresponding to 30 rail sections are determined for each 5-s vibration segment. b The location map of a downward high-speed train. Dotted line shows when the train reaches each of the 30 rail sections, among which typical infrastructures are highlighted: Section 7 is ballasted track bridge; Section 13 is ballastless track roadbed; Section 21 is the ballastless track bridge. Segments after Section 30 are vibrations occurring at rail section out of the 12-km sensing line. Their vibrations propagate back and are detected by the laser interferometer.

In terms of the location labeling from Res-LstmNet and subsequent field investigation, we associate these 30 labels with actual HSR sections and form a location map. The map allows us to collect and analyze vibration data for each section, to assess its health condition. For instance, Section 7 corresponds to a ballasted track bridge - Lianyu Bridge. Its actual photo is shown as the inset I in Fig. 1, and the train-induced vibration is marked in Fig. 2b. By extracting and examining vibrations of Section 7 at different times, we can find its evolution process and assess its health condition. Similarly, we also select other typical sections for further analysis: Section 12 corresponds to the traditional ballasted track roadbed, Section 13 turns into the ballastless track roadbed (shown as inset II in Fig. 1), and Section 21 is the section of ballastless track bridge over the Yongding River (shown as inset III in Fig. 1). Taking Section 13 as the dividing point, we can see that the 12 km HSR line can be mainly divided into the ballasted track part and ballastless track part. The ballasted track, particularly in urban area, can largely eliminate the train-induced vibration and contribute to a “quiet” background, while the ballastless track will induce more noisy vibration background. Exceptions occur when the rail goes over bridges in urban area (like Section 7), which cause obvious vibration peaks. Out of the city (Section 14 ~ 30), high viaducts are commonly employed. These 4 kinds of sections (ballasted track bridge, ballasted track roadbed, ballastless track bridge, ballastless track roadbed) can be used as examples to analyze some typical scenarios along the HSR.

In Fig. 2b, another interesting phenomenon is observed after 329 s. The high-speed train has been out of the sensing rail line, while vibration signals are still detected by the laser interferometer. In fact, the vibrations detected afterwards are actually excited by carriage passage on the adjacent 2 km stretch of railway, and propagate back through the HSR viaduct. This scenario clearly shows the loading-unloading process of carriage passage and detailed analysis is provided in Supplementary Note 1.

HSR health inspection using A-PSD

Based on the location map given by the Res-LstmNet, we can obtain the power spectrum density (PSD) of each HSR section. Taking the ballasted track bridge – Lianyu Bridge at Section 7- as example, in Fig. 3a, the loading-unloading process of 8-carriage train can be seen in time domain, which leads to fundamental frequency and harmonics distributed at integral multiples of 1.5 Hz in Fig. 3b (corresponding to 25-m carriage length). Similarly, 2.5-m bogie axle passage will also induce vibration components, with frequency of integral multiples of 14.6 Hz. Besides these train character related vibrations, track and roadbed also contributes to vibration spectrum, because vibrations occurring at wheel-track interface will propagate through track bed, and take in information of the whole TTB system^42,43,44,45. Actually, the train, the track and the bridge form an integrated dynamic system, in which the train and bridge are coupled by the wheel–track interactive relationship. Therefore, the bumps of PSD in Fig. 3b can reflect characteristics of the TTB system: the spectrum bump at 1 to 10 Hz is mainly caused by carriage loading-unloading process on the track, in other words, is the carriage-track coupling natural frequency of TTB system; the bogie-track coupling natural frequency contributes to the next spectrum bump at 10 to 20 Hz; and the spectrum bump around 60 Hz can be regarded as natural frequency of track irregularity to a great extent⁴⁶.

Fig. 3: The average power spectrum density (A-PSD) analysis of 4 typical HSR infrastructures.

a The vibration waveform of train passing the ballasted track bridge (Lianyu Bridge at Section 7). b The PSD plot of ballasted track bridge Section 7. c The A-PSD of the ballasted track bridge Section 7. d The A-PSD of the ballastless track bridge Section 21. e The A-PSD of the ballasted track roadbed Section 12. f The A-PSD of the ballastless track roadbed Section 13. g The comparison between A-PSD mean value of ballasted/ballastless track bridge. h The comparison between A-PSD mean value of ballasted/ballastless track roadbed. A-PSDs are collected on 4 dates (Feb. 22nd, Aug. 1st, Aug. 6th in 2023, Apr. 16th in 2024). The overall error range of A-PSD is described as gray shade, calculated by 2 σ range.

To focus on railway fault inspection and reduce the false alarm rate, we calculate the average PSD (A-PSD) of 30 passing trains on each HSR section. Figure 3c gives the A-PSD plot of Lianyu Bridge on 4 dates with a time span over a year (Feb. 22nd, Aug. 1st, Aug. 6th in 2023, Apr. 16th in 2024). During the time, seasons change from winter to summer with a 50-°C variation, the heaviest rainstorm in past 140 years hits the city starting from Jul. 29th 2023, and an earthquake of 5.5 Mw occurring 300 km away strikes the HSR system. Although it suffers violent changes above, the A-PSDs remain consistent and the HSR system are in a good health condition all the time. An error range of all measured A-PSD is obtained (shown as the gray shade in Fig. 3c), which covers 2 σ range (2 standard deviation) to indicate the margin of healthy state.

Apart from the ballasted track bridge above, the A-PSD of 3 other infrastructures (ballastless track bridge, ballasted track roadbed, and ballastless track roadbed) are also recorded. The A-PSDs of each infrastructures are consistent as shown in Fig. 3d–f. However, they show different characteristics from each other. In Fig. 3g, we compare the A-PSD mean value of bridges on ballasted and ballastless track. The characters in low frequency (1 ~ 20 Hz) are similar to some extent, because they mainly reflect train-track coupling features. But difference becomes larger above 30 Hz due to their different structure characteristics. The ballasted track bridge has the strongest excitation peak around 60 Hz, while the peak of ballastless track bridge appears at 30 to 50 Hz. In Fig. 3h, we compare the A-PSD mean value of roadbeds on ballasted and ballastless track. Roadbed has a significant attenuation to train vibrations, but still fluctuates in the frequency band of 30 ~ 70 Hz due to the track-wheel interaction. Therefore, the train-induced A-PSD has potential to be used as the fingerprint of each infrastructure on HSR line. The change in A-PSD may give warning of track faults, which may greatly narrow the inspection period from once a month to dozens of times a day. It should be noted that the spatial resolution of the A-PSD is ~ 400 m, which is in accordance with the length of 16-carriage high-speed train. In practical engineering applications, a spatial resolution of 400 m already provides sufficient positional reference for further railway defect identification and maintenance, making the fiber sensing system an effective supplement between two times of TRV inspections.

A-PSD analysis of creep deformation on HSR bridge

To demonstrate the A-PSD response to those possible track faults, we find a section with subtle creep deformation and carry out deeper analysis. Actually, the creep deformation is inevitable, and the whole line is in a healthy condition^47,48. The chosen section, for instance, corresponds to the bridge span over Beijing Winter Olympic Park and has 3 mm creep deformation at first. It is recorded by TRV inspection on Mar. 28^th 2024 and shown as the blue curve in Fig. 4a. According to China’s regulations on high-speed railway subgrade design and maintenance (TB 10621-2014, TG/GW 120-2015), this creep deformation is far from health warning, but the HSR Engineering Section of Beijing Railway Bureau still carries out track maintenance to enhance the track smoothness. From Apr. 9th to Apr. 13th in 2024, a series of maintenance works are implemented. As a result, the creep is nearly eliminated which is shown as the red curve in Fig. 4a (TRV data recorded on Apr. 16th 2024).

Fig. 4: Sensing results of Section 19 before and after maintenance.

a The vertical response detected by track recording vehicle train before and after maintenance (with a time interval of ~20 days). b The average power spectrum density detected by fiber sensing system of the same section. The difference of HSR response can be immediately acquired after maintenance.

In fiber sensing A-PSD, the difference can also be observed. Figure 4b presents A-PSDs of this section before and after track maintenance (Apr. 8th and Apr. 9th 2024). The 3-mm creep indeed causes spectrum bumps in 2 frequency bands: ~20 dB rising at 2 to 5 Hz and ~15 dB rising around 80 Hz. These two frequency bands correspond to carriage-track coupling band and track irregularity band. The reason is that the frequency response of TTB system is not isolated. The creep can be regarded as additional irregularity of rail track, which corresponds to the rising peak around 80 Hz, and will also induce periodic excitation of carriages in low-frequency band. As a result, the laser interferometer is efficient for creep deformation inspection with higher response speed. It narrows the period of inspection from nearly 20 days (TRV) to the immediate response after maintenance. During the observation time, another subtle creep deformation is observed and analyzed in Supplementary Note 4. In fact, some other defects, such as frost heaving and slab expansion, will induce stronger irregularity in track geometry^49,50, thus should be recorded and noticed by fiber vibration sensing system. Besides the creep deformation sensing comparison, a qualitative and indirect comparison between fiber sensing vibrations and TRV inspection data is also provided in Supplementary Note 2.

In railway engineering practice, the A-PSD with error margin can serve as a unique fingerprint for each section of the HSR line, indicating the boundaries of the healthy operational state. The A-PSD can be calibrated by the TRV inspection result at least once a month. When the A-PSD of a section significantly deviates from the allowable error margin, it indicates potential risks in the infrastructure of that section. This necessitates a manual railway inspection to further identify the specific issue. It should be noted that, to focus on railway fault inspection and reduce the false alarm rate, the averaging operation of the A-PSD smooths out specific vibration characteristics of individual trains, and preserves more features that reflect the characteristics of the railway infrastructure under train-induced excitation. Therefore, the A-PSD is insensitive to train faults, and mainly focus on the railway faults, such as track fractures, track heaving, ballast settlement and so on. In the future, as more types of railway faults are detected through fiber sensing, a mapping relationship between A-PSD response changes and types of railway faults will be established, making fiber sensing increasingly intelligent.

Ambient vibrations detected by HSR fiber sensing system

Apart from high-speed train induced vibrations, there are some unexpected vibrations occurred and recorded, especially at night. During the nighttime maintenance window from around 0:00 a.m. to 5:00 a.m., no high-speed trains will run on the railway, creating a quiet environment that allows the detection of other ambient vibrations.

Among all of the detected vibration events, the most violent ambient vibration is caused by a Mw 5.5 earthquake, occurring at 2:33:59 a.m., Aug. 6^th, 2023 in Shandong Province. The earthquake propagated over 300 km, struck the HSR and was detected by the fiber sensing system. Benefiting from the quiet environment, the seismic wave detection has high signal-to-noise ratio (SNR) shown in Fig. 5: SNR of the first arrived primary wave (P wave) is better than 10 dB, and SNR of the stronger secondary wave (S wave) can reach the maximum value of 20 dB. Therefore, the arrival time of P wave and S wave can be clearly found and the time difference of arrival (TDOA) between them is ~35 s. It is reasonable when considering the epicenter distance as 326 km, the approximate seismic wave speed of P and S waves as 6.1 km/s and 3.5 km/s in upper crust^51,52.

Fig. 5: Seismic wave detected by the fiber sensing system.

The Mw 5.5 earthquake occurs 326 km away and propagates to the high-speed railway in Beijing at night. P wave primary wave, S wave secondary wave.

Another interesting ambient vibration is also observed at night. We find a series of vibrations occurring at a specific location, 6.9 km away from South Shawo Bridge. After field investigation, a normal railway line is found at the spot, going under the high-speed viaduct in perpendicular direction, the map is shown as Fig. 6a. Coal trains travel along the line and pass the cross point at night. The induced vibrations will propagate along the viaduct pier and up to HSR to be measured by fiber cables. Since there is attenuation during the propagation, these detected vibrations are not much stronger than background noise as shown in Fig. 6b. But in frequency domain, vibrations have distinctive characters and can be easily distinguished, as shown in Fig. 6c. Time-frequency curve of each order harmonic goes with the transient speed of train, which is similar to the result of high-speed trains (Supplementary Note 2). The speed changing rule of coal train can be clearly observed from those time-frequency curves (red curves corresponding to 3 trains).

Fig. 6: The sensing results of ambient vibration caused by coal train passing under the high-speed railway system.

a The cross point diagram and map of the normal railway and high-speed rail. b Three coal trains induced vibrations detected by fiber sensing system. c The short-time Fourier transform spectrum of the ambient vibration results. Those speed-related time-frequency curves are highlighted as red curves.

a Short-time Fourier transform (STFT) of vibrations obtained by laser interferometer, with size 36×79 every 5-s segment. During the period, three high-speed trains from two different directions pass the sensing railway. b Location labels obtained by distributed acoustic sensing, with size 1×30 every 5-s segment. Phase values are normalized and can be seen as the true label of Res-LstmNet.

Details of the Res-LstmNet

The Res-LstmNet aims to locate the detected train-induced vibration. Each HSR infrastructure section (in our case, 400 m high speed railway) has its specified vibration pattern- represented by the A-PSD, which can act as the fingerprint of each HSR section. In normal case, it remains stable on the same HSR section, and differs on different sections. The proposed Res-LstmNet identifies distinct vibration characteristics and matches them with the patterns of different HSR sections. The input of Res-LstmNet is the short-time Fourier transform (STFT) matrix of vibrations obtained by laser interferometer. The expected output is the location label obtained by DAS. The two sets of data used to train the Res-LstmNet are collected simultaneously. The dataset contains 148,700 s of data collected over the course of one week. Then the data is divided into 29,740 segments, each with the length of 5-s. The segments are then converted into STFT spectrum, with the window length of 1 s and step of 0.1 s. Each STFT segment is a 2D matrix with a 5-s duration, consisting of 36 effective time points (with 2 points removed from both sides to reduce the correlation between adjacent segments) and 79 frequency components covering the 1–79 Hz range. The STFT segments act as the input of neural network. DAS data is arranged as time-location arrays, with size of 1×30 every 5-s segment, and noise is removed through moving average and thresholding operation. Training set has 28000 segments of data with 5 s length, and test set has 1740 segments, with a ratio of 16:1.

Two parts of neural networks are used in Res-LstmNet. First, each 5-s STFT matrix with size 36 × 79 goes through a ResNet⁵⁶ to extract its feature. The size of each ResNet output is 1 × 1024. The current output is combined with the previous 99 continuous outputs to form a time-domain sequence with size 100 × 1024. Each sequence goes through the second Lstm Network⁵⁷. In the Lstm Network, there are two intermediate variables to record long-term and short-term memory, respectively. These 100 components in the time-domain sequence are sent into the Lstm Network one by one, and the recording intermediate vectors are renewed step by step, accordingly. Finally, the network gets one output with size 1 × 30, in accordance with the number of rail sections, representing the location result for the current input vibration signal. The overall process of prediction is shown in Fig. 8.

Fig. 8: The structure of Res-LstmNet.

After pre-processing, a time-continuous sequence of 100 short-time Fourier transform pictures go through a Residual Neural Network and get a characteristic sequence with size 100 × 1024. Then components of the sequence go through the Long Short-Term Memory network one by one. Long-term and short-term memories are changed accordingly. Finally, after the 100th component goes through the network, a result vector with size 1 × 30 represents the localizing probability of the 30 rail sections. The predicted section can be extracted by finding the maximum probability point. STFT short-time Fourier transform, Conv Convolution layer, ReLU Rectified Linear Unit, Lstm Long short-term memory.

During the training process, the training set is shuffled and is traversed for 100 epochs. The parameter of network is randomly initialized and updated every segment in training set. The Adam optimizer is used. The loss function is shown as follows,

where ??? is the label, ??? is the network prediction result, N is the total number of STFT segments, W contains all weights in the Res-LstmNet, M = 30 is the number of sections, ?1 and ?2 are variable parameters. The first term represents cross entropy loss between network output and the DAS localizing label. The second one is a regularization term to avoid the parameters in the network to become too large, which can also help to avoid overfitting.

The input 5-s STFT matrix carries its own timestamp. Consequently, the corresponding localization results can be automatically ordered. The ordered localization results form a time-location waterfall plot based on timestamp, representing the location of vibration at each time, for example the localization result in Fig. 2a.

To demonstrate the performance of the Res-LstmNet, the localization results on test dataset of 1740 segments (~27 trains) are shown in Fig. 9a, and the expected DAS results are shown in Fig. 9b. By comparing the predicted localization results with those from DAS, a multi-label confusion matrix^58,59 is obtained, as shown in Fig. 9c. Row 0 represents there is no train on the track, while rows 1 to 30 represent there are trains on these sections, respectively. Column 0 represents the Res-LstmNet predicting no train on the track, while columns 1 to 30 represent the Res-LstmNet predicting trains on these sections, respectively. The bright spots mainly distribute on the diagonal line, indicating that the results of Res-LstmNet are generally consistent with the DAS results. We also observe some bright spots in column 0 (excluding the point [0,0]), indicating that approximately 21.1 % of trains are missed by Res-LstmNet. Most of these missed detections occur near Section 30, and the reason is discussed in Supplementary Note 1. In fact, these missed detections are harmless and do not affect the accuracy of the A-PSD results, but only reduce the utilization rate of the detection data to approximately 78.9 %. In contrast, the bright spots in Row 0 (excluding the point [0, 0]) are harmful, as they indicate that the Res-LstmNet falsely predicts a train where there is none. However, this false positive rate is very low, at only 0.1 %. For all effective data (78.9 % of the detection data), the prediction accuracy of the Res-LstmNet is approximately 89.8 %. Taking into consideration the results mistaken with only 1 section, the prediction accuracy can reach 99.5 %. If the data of DAS from sections 29 and 30 are excluded from Res-LstmNet’s training process, the prediction missing rate will be reduced from 21.1 % to approximately 15.2 % (in Supplementary Note 1). The prediction accuracy of the effective data remains relatively consistent.

Fig. 9: Localizing results and the multi-label confusion matrix of Res-LstmNet on test dataset of 1740 segments.

a Localizing result of Res-LstmNet, presenting the time-space distribution of passing trains. b Location labels obtained by distributed acoustic sensing (DAS), which is the expected output. c Multi-label confusion matrix comparing predicted localizing result with DAS result.

When extreme events (earthquake or accident) cause fundamental changes in the roadbed structure, the overall trend of the STFT spectrum used for localization may change significantly. Then the neural network may lose its localization capability. In such cases, recalibration of the neural network using DAS localization results is required. In practice, real-time localization monitoring results (Supplementary Movie 1) can be used to determine whether the recalibration is necessary. Specifically, when the neural network’s output exhibits noticeable discontinuity that contradicts the expected movement pattern of the train, the latest STFT spectrums and DAS localization data should be used to recalibrate the neural network.