Aiming at the problems that the traditional detection method can′t detect and see the target clearly, and the image contour and detail are blurred. The combination of infrared and polarization detection is adopted to solve the problem that the infrared image information and polarization image information cannot be detected and seen in various environments. Aiming at the problems of large amount of data and slow extraction speed in the process of target local feature extraction, an improved deep learning local feature extraction (SIFT) algorithm for polarization images is proposed. Experimental results show that the improved algorithm combines the advantages of polarization imaging and deep learning to achieve rapid feature extraction of targets in simple or complex backgrounds. This algorithm improves the speed and accuracy of local feature extraction of polarized images. The improved algorithm lays a theoretical foundation for target classification, recognition and tracking technology.