Lidar is widely used in underwater detection. However, laser is scattered during underwater propagation, and scattering is an important factor limiting the performance of radar systems. Lidar-radar techniques can suppress both forward and backward scattering, where the cut-off frequency of the scattered light is an important parameter. However, unlike backscattering which is easily separated from the target-reflected light in space, forward scattering is difficult to separate from the target-reflected light as they are intertwined in space. Therefore, we propose to separate the forward scattered light from the target-reflected light using vortex beams and analyze its frequency response to obtain the cut-off frequency of forward scattered light.Vortex beams are used to distinguish between forward scattered light and target-reflected light in underwater target detection (Fig.3). The fast Fourier transform is done on the spatially filtered forward scattered light and the target-reflected light to obtain their frequency response (Fig.4). A Monte Carlo-based model for lidar-radar underwater target detection is also developed, and the fast Fourier transform of the forward scattered and reflected light in the echoes is also performed to analyze their frequency response (Fig.1). The simulation results are compared with the experimental results (Fig.2, Fig.6). A ranging experiment is designed to investigate the effect of modulation frequencies less than or greater than the cut-off frequency of forward scattering on the accuracy of ranging (Fig.7).The experimental results show that scattering has an averaging effect on the modulation, and the high-frequency component is difficult to be maintained in the forward scattering. On the contrary, the high frequency modulation is well maintained in the signal light (Fig.4). The higher the modulation frequency is, the higher the signal-to-clutter ratio is. In addition, in order to achieve a signal-to-clutter ratio of 1, higher modulation frequencies are required in more turbid water (Fig.6). The experimental results are consistent with the results of Monte Carlo simulations (Fig.2). The results of the ranging experiments show that when the signal-to-clutter ratio is less than 1, increasing the modulation frequency reduces the ranging error significantly. While when the modulation frequency is high enough to make the signal-to-clutter ratio greater than 1, continuing to increase the modulation frequency makes the ranging error decrease roughly, but not change much (Fig.9).A method for spatially distinguishing forward scattered light in underwater target detection echoes using vortex beams is proposed. Thereby, the forward scattered light and the target-reflected light are obtained separately in the experiment, and the frequency response of both is analyzed by fast Fourier transform to calculate the modulation frequency sufficient to suppress the forward scattered light in the lidar-radar technique. The frequency response of forward scattered light and target scattered light in underwater target detection is simulated based on Monte Carlo method and compared with the experimental results. Both simulation and experimental results show that a high modulation frequency can improve the signal-to-clutter ratio of the underwater lidar system. And for turbid water, a higher modulation frequency is required for more effective suppression of forward scattering. To achieve a signal-to-noise ratio greater than 1, modulation frequencies greater than 700 MHz are required for an attenuation length of 11 and greater than 900 MHz for an attenuation length of 12. Ranging experiments also demonstrate that when the modulation frequency is not high enough to make the signal-to-clutter ratio greater than 1, increasing the modulation frequency results in a significant reduction in the ranging error. While when the modulation frequency is high enough to make the signal-to-clutter ratio greater than 1, continuing to increase the modulation frequency results in a general reduction in the ranging error. When the modulation frequency is high enough to make the signal-to-clutter ratio greater than 1, the ranging error decreases roughly, but the change is not significant. The findings of this study can be used as a reference for the design of carrier modulated underwater lidar systems.