To address the problem of target loss facing existing video tracking algorithms due to high mobility of targets or rapid deformation of asymmetric rigid targets, this paper proposes a video tracking algorithm based on the correlation filtering adaptive model and the redetection mechanism for average peak-to-correlation energy (APCE). The adaptive model tracking algorithm can adjust the model in real time according to the clarity of the target area, thereby effectively ensuring the accuracy of the target tracking model. Experimental results show that integrating the adaptive model tracking algorithm into the discriminative scale space tracking (DSST) model can enhance the tracking effect of the model on highly mobile or rapidly deforming objects. While guaranteeing tracking speed, the integration also raises the average accuracy of the original DSST model by 18.3 percentage points and the success rate by 15.2 percentage points. In addition, combining the adaptive tracking algorithm with the APCE redetection mechanism can ensure the stability of the algorithm.