Infrared target tracking is of great significance to the research in military and civil video surveillance. Due to the special thermal-imaging mechanism, infrared targets are often with low resolution, low contrast and in the lack of textures. Aiming at the deterioration of tracking performance caused by insufficient common features of infrared targets, a novel tracking algorithm was proposed based on adaptive fusion of correlation filter responses. The algorithm explored the framework of the correlation filter with continuous convolution operators. The saliency feature was comprised to enhance the object appearance description. The center location of a target was predicted by the fused responses that were calculated from an adaptive fusion of multiple correlation responses with Hedge decision-theoretic. Additionally, the final tracking result was obtained after multi-scale estimation based on scale filters. The experimental results show that the algorithm has better performance in tracking accuracy and robustness compared with other tracking methods on the public infrared video dataset VOT-TIR2016.