The adaptive filtering algorithm based on recursive least square algorithm was applied to a laser Doppler vibrometer and the corresponding weak vibration measuring devices were built up. Compared with Chebyshev low-pass filtering algorithm, simulation and experimental results show that this adaptive filtering algorithm can suppress the random Gaussian white noise and restore the original signal; the RLS algorithm can effectively filter out noise from the vibration signal, and restore the low-frequency 20 Hz signal drowned out in the noise; the RLS algorithm also can filter out noise from voice leading voice to being pure and enhance speech signal. The above verifies the feasibility of this algorithm in the heterodyne vibration measurement. The algorithm is simple and easy to use, fast and has strong convergence. In the aspect of filtering random noise, the RLS algorithm is more effective than low-pass filter.