Infrared polarization imaging has potential advantages in anti-jamming target detection and man-made object recognition in complex environment, and can obtain the physical and chemical characteristics of the target surface. The application of division-of-time, division-of-amplitude and division-of-aperture infrared polarization imaging technique is limited to the insufficiency of volume, weight and power consumption. Miniaturization, integration and real-time imaging equipment are the premise of the widespread application of infrared polarization imaging. Intelligent analysis of the acquired data is the basis of its application. In this paper, a new developed infrared polarization intelligent sensing system was introduced, which collected the infrared polarization data of the target scene in real-time by using the division-of-focal-plane imaging technology. By combining the deep learning with the division-of-focal-plane polarization imaging closely, the high quality polarization image restoration and the intelligent sensing of moving target in typical scenes were realized.