To deal with the problem that traditional real-time correlation tracking methods are sensitive to the light variation, a Phase Congruency(PC) image based on tracking method is proposed which utilizes the properties of PC function falling in ［0, 1］ interval, being normalized and invariant to changes of image brightness and constrast. PC detection is performed on the original image, and PC image is obtained. Subsequently, target tracking with correlation tracking methods, such as Minimum Absolute Difference (MAD) is conducted on the PC image. Experiment results show that the proposed method can achieve a satisfactory tracking performance on visible light images and infrared images even though the image brightness and constrast vary abruptly.The phase congruency image based on tracking method can be used to solve the common problems of tracking point drifting, even failed tracking due to image light variation in traditional real-time correlation tracking methods.