An improved tracking algorithm is presented in order to solve the problem that traditional tracking algorithm based on Mean Shift often falls into local extrema and causes inaccuracy or even failure when tracking a camouflaged target. Firstly, joint histogram is used to enhance the description of camouflaged target’s character. Secondly, target description model is modified according to the discrimination between the target and local background during the tracking period. Finally, the weights of each part of the joint histogram are adjusted adaptively to ensure more accuracy and robustness to background motions. The updated algorithm is tested with image sequences where the colors for both the target and the background are similar and results show that it can achieve effective and exact tracking.