For the problems of tracking drift or loss in the tracking algorithms of correlation filter, which is caused by fast motion, occlusion and appearance variation of objects, an object tracking algorithm with kernel cross-correlator based on temporal consistent constraint is proposed. A kernel cross-correlator vector, which is more robust to image noise and clutters, is introduced to predict affine transformation of object more precisely. Meanwhile, to solve the problem of tracking drift caused by temporal degradation of kernel cross-correlator, temporal consistent constraint is introduced during learning process. Finally, MGC(major gray component) anti-projection is utilized to improve the ability of the tracker to deal with occlusion of object. The proposed algorithm is compared with provided benchmark algorithms and other more advanced correlation filter based algorithms on public OTB100 standard object tracking datasets. The tracking speed of proposed algorithm precision reaches 41f/s, and compared with fDSST and SAMF algorithm, its tracking precision is increased by 15.6% and 6.4%, and the success rate is increased by 33.3% and 6.1% , respectively. The experimental results demonstrate that the proposed algorithm is able to track objects precisely under the condition of fast motion, occlusion and appearance variation.