To effectively evaluate the tracking ability of a photoelectric theodolite, a new tracking error model based on the Radial Basis Function(RBF) neural network was established. First, the nonlinear factors existing in the theodolite were described and the reason why the system was hard to be modeled based on theory was discussed. Then, the RBF neural network theory and the target system were introduced, and the RBF neural network model was built and verified in different parameters. Finally, the network model with new parameters and data was trained and the new network model was obtained through changing parameter periods. Experimental results indicate that the precision of the neural network is closely dependent on the target system parameters. When the half cone angle(a) and the tilt angle(b) of a dynamic target are 21.2° and 43.8°, respectively, and the moving period(T) is 8.5 s, the maximum model error is 3.18′ in the acceleration coming to the maximum. And for other time, the model error is less than 0.6′. Furthermore, when the a and b are 16.6°, 37.5°, and T is 13 s, the maximum model error is about 1.8′. With the network model, the maximum error between model output and real output is 2.4′ in the speed coming to maximum. And for other time, the maximum model error is less than 1.2′. The results indicate that the network model based on RBF neural network can replace a real system in a certain sense. It is feasible and has high accuracy and important value to the engineering practice.