A high-confidence adaptive feature fusion target tracking algorithm is proposed to address the problem of tracking drift in the complex scenes, such as occlusion and complex backgrounds, using a complementary learning tracking algorithm. First, we use the Bhattacharyya coefficient to calculate the similarity between the foreground and background color histograms of each frame in real time, and adopt the log loss function to obtain the final fusion factor to achieve better feature fusion of each frame. The average peak correlation energy and the ratio of the response peak value to its corresponding historical average value are then used to determine confidence determination parameter, and the target position is updated and corrected for tracking based on the determination result. Experiments on the OTB100 and LaSOT datasets show that this algorithm improves the precision rate by 17.5% and 15.4%, and success rate by 27.3% and 18.0%, respectively when compared with the Staple algorithm. The results demonstrate the effectiveness and robustness of the algorithm.