Aiming at the problems of high dimensionality, weak stability, and poor registration quality of the features to be matched caused by the difference in sensor physical characteristics in heterogeneous image registration, this paper proposes a scale-invariant feature transform (SIFT)-based heterogeneous image registration method. This method combines the phase consistency and improved SIFT algorithm to obtain stable features. Next, it uses the nearest neighbor distance ratio method for initial matching. Then, we propose a joint error and Euclidean distance (JEED) method for rematching. The mode-seeking scale-invariant feature transform (MS-SIFT) method is employed to optimize the matching point pairs to improve the image registration quality. Experimental results show that, compared with the existing methods, the method proposed in this paper can extract reliable and stable features, obtain higher registration quality, and improve the real-time performance of the registration algorithm.