To track the moving target in the cluttering backgrounds is one of the most important research fields in computer vision.A new tracking method based on the significance weighted least square image matching was proposed.According to the variance between the target and the background,the new method set the different weights to every parts of the target and tracks the target by the Significance Weighted Least Square Image Matching Method (SWLSIM).We compared our new method with the three classical tracking methods.Experimental results show that the new method can track moving targets in the cluttering background effectively and accurately.However,the tracking results of the other three methods are unacceptable.And the new method is satisfied for real-time tracking processing.