Based on the theory of the Fresnel polarization reflectance, the polarization factor was put forward by decomposing the incident light into electric vector, which could affect the intensity of glare and play a leading role during eliminating glare. According to the imaging detection model, the displacement factor was presented, which could change the glare position and the size of the glare area, and play a supplementary role during eliminating glare. The image gray-scale transformation curves under different polarizations and different positions were gained by sampling and contrasting the image datas of a certain object, and these image data were measured in the equivalent environment while different situations. The proposed theory was verified by analysing the relationship of gray-scale transformation curve with polarization and position. The results show that the essence of glare eliminating is reducing the light intensity and contrast between with the glare area and neighborhood region. Furthermore, the joint mobilization of the displacement factor and polarization factor can almost eliminate the glare completely completely.