A multiscale fusion algorithm based on image enhancement and rolling guidance filtering is proposed to solve the problems of thermal target brightness loss and visible image detail information loss caused by infrared and visible image fusion. First, an adaptive image enhancement method is proposed to improve the overall brightness of the visible image and maintain the contrast of the details. Second, according to the different features, the source image is divided into three layers, and the luminance layer is obtained by using the significant extraction method based on guidance filtering. The favorable scale perception and edge preservation characteristics of rolling guidance filtering are used, and the basic layer and detail layer are obtained by combining Gaussian filtering. Finally, the fusion rule of large pixel value is used for the luminance layer, a least-squares optimization scheme is proposed for the basic layer, and the sum of the modified Laplace energy is used as a measure of sharpness for the detail layer. The experimental results show that, compared with other fusion methods, the proposed method has better performances in both subjective and objective evaluations.