A natural global color mapping method matches the statistical properties of night vision imagery to those of a daylight color image. Thus the rendered night vision image appears to resemble the natural target image in terms of color appearance and the reference image is not automatically selected. However, the colored night vision image may appear unnatural if the target image’s global color statistics are different from that of the night vision scene. And then target is not easy to be detected because the color of target and background is similarity. Consequently, a new local color transfer method was presented. Firstly, the infrared(IR) and low-level-light(LL) images were fused by using sparse representation. Secondly, a nonlinear diffusion filter was applied to the gray fused image. The diffused image segmented by Kernel fuzzy C-mean algorithm. Finally, the IR and LL images block were color fused by using color transfer technology in the YCbCr color space, where the Y component of the color image was replaced with the fused image, which is a gray scale image made with multi-band night vision images. Experimental result shows that the obtained image has more natural and realistic coloration compared with global coloring method.