Traditional sun-photometer cannot accurately track the sun on offshore mobile platform. In order to solve the problem of high-precision tracking of the sun in the process of ship moving, an image tracking system for shipboard sun-photometer(SSP) was constructed by using fisheye lens, gyro-stabilized platform and small-field CCD image sensor. The whole structure of the image tracking system of the shipboard sun-photometer and the design of the optical path of the single-arm probe were introduced. The coarse tracking of the sun in the large field of view by the clock method and fisheye imaging system was described in detail, and then the fine tracking measurement was carried out by the CCD image processing technology of small field of view. Besides, the software tracking algorithm, process and system tracking reliability were given. The image tracking system of shipboard sun-photometer can be automatically tracked and measured on offshore mobile platforms, and the comprehensive tracking accuracy was higher than 1′. The data were compared with the Japanese POM-01 MKⅢ sun-photometer. The results show that the maximum relative error of atmospheric transmittance at 940 nm band is less than 7.6%, and the maximum relative error of water vapor is less than 6.1%. The system can be applied to the shipboard sun-photometer to measure the atmospheric transmittance and water vapor at sea, and can also be used to track the sun on other moving unstable platforms.