According to tumor features under gastroscope, the template matching tracking algorithm based on speeded up robust features (SURF) are adopted to track the tumor image. Mismatching points of features are removed to improve the tracking accuracy. In matching tracking algorithm based on SURF, the clustering center of matching feature points and smallest circular position around the feature point are used to measure tracking effect of each frame. Video frames of two groups of gastroscopic lesions are regarded as experiment dataset for target tracking tests. Results show that the improved matching tracking algorithm based on SURF has good stability and tracking accuracy.