A template drift suppression method based on sub pixel correction is proposed to address the issue of long-term template drift tracking by the kernel correlation filtering algorithm. First, according to the fast algorithm of kernel correlation filtering commonly used in the tracking process, the tracking error introduced by template updating was analyzed. Second, the influence of sampling error on template drift was analyzed. Then, the sampling error introduced by the template update was corrected by shifting the phase of the template frequency domain. Finally, the inhibition effect of the template sub pixel correction on the long-term tracking template drift was verified through experiments. The experimental results show that under the same tracking conditions, the template drift introduced using the template sub pixel correction method is reduced to only 1/4 of that of the whole pixel template update. Therefore, the template sub pixel correction quite effectively suppresses the template drift with a small amount of computation when used in the correlation filtering and kernel correlation filtering algorithms.