Most existing feature point tracking algorithms only consider two adjacent frames at a time and neglect the feature information of previous frames. In this paper, based on the original Kanade-Lucas-Tomasi (KLT) algorithm, a new feature tracking method is presented that learns an eigenspace representation of training features online, and finds the target feature point with Gauss-Newton style search method, effectively avoiding the troubles introduced by target dimension, large angular changes and occlusion. By analyzing the performance of the algorithm by the actual image, it is compared with other algorithms. Meanwhile, we design a hardware platform using the eight-core digital signal processor (DSP) TMS320C6678 as the core processor, in combination with field programmable gate array (FPGA) digital video image acquisition device. A series of strategies of the algorithm optimization are proposed. Finally, tracking of moving target is achieved by the realtime transmission of video images.