The measurement of micro-vibration signals of objects has important application value in magnetic field, construction, biological imaging and aerospace. However, the weak reflected light generated by the micro-vibration of the object is not only extremely weak, but also susceptible to interference from environmental factors such as detection distance and rain and fog. Moreover, the vibration form of low-frequency vibration is variable and susceptible to classical noise. It is difficult to achieve vibration signal measurement under extremely weak reflected light conditions. In view of the above problems, this paper realizes the measurement of micro-vibration signal in Hertz frequency band and weak reflected light based on polarization control.Optical polarization control method is used to control the polarization of signal light and local oscillator light to reduce the interference of optical noise. Balanced heterodyne detection is used to convert low-frequency DC signals into high-frequency AC signals to avoid signal annihilation by noise (Fig.1). The power spectral density (PSD) analysis of the photocurrent formula output by the balanced detector is carried out by MATLAB, and the relationship curve between the ratio of the noise power of the vibration signal to the power of the heterodyne signal and the vibration displacement is obtained (Fig.2). Then, the corresponding value is obtained according to the power spectrum of the vibration signal and the heterodyne signal in the Hertz frequency band measured by the experiment (Fig.9). Finally, the measured value of the micro-vibration displacement is obtained.In the Hertz frequency band, the noise level of the detector reaches the shot noise limit (Fig.7), and the micro-vibration signal measurement under the condition of Ava-level reflected light is realized. When the PZT load voltage is 1 Vpp, the optical power of the input signal light is 1.06 × 10¹⁸ W, the micro-vibration amplitude of the object is 11.44 nm, the class A standard uncertainty is 0.25 nm, the combined uncertainty is 0.34 nm, and the measurement accuracy is ± 0.75 nm (Tab.2). A balanced heterodyne detection method controlled by optical polarization is used to measure the shot noise limit of object micro-vibration signals in the frequency range of Hertz (~10 Hz). The minimum amplitude is 11.44 nm, the standard uncertainty of class A is 0.25 nm, the combined standard uncertainty is 0.34 nm, the measurement accuracy is ± 0.75 nm, and the noise level reaches the shot noise limit. This scheme not only provides experimental support for the research of weak Doppler frequency measurement, low frequency vibration signal detection and other measurement fields, but also has broad application prospects in complex measurement environments such as weak reflected light, long distance, rain and fog weather.