Information entropy weighted by image variance is a robust quantitative indicator describing the complexity of image. It can achieve good results by using information entropy weighted by image variance to detect the infrared small target. However, it is difficult to apply in engineering application due to its complex calculation and poor real-time performance. To recognize the small target regions under infrared complex sky background quickly, we improve the traditional image filtering algorithm, which uses information entropy weighted by image variance. Images are segmented according to their saliency first. Significant regions are selected roughly, and only the information entropy weighted by image variance of the dual-mode regions of the salient regions is calculated. Then, the candidate target regions are recognized according to the typical regional features of the information entropy weighted by image variance of the dual-mode regions under complex sky background. The experimental results show that the proposed algorithm can eliminate the disturbance of the complex sky background, and reduce the running time of the algorithm.