An improved method to fit the side wobble trace of a positioning setup to compensate the software was proposed to reduce the error from the mechanical swing of the focusing setup during focus plane adjustment.The alignment mark was positioned through the sub-pixel correlation template matching algorithm and the validity of the algorithm was investigated, which shows that the resolution of the algorithm is better than 0.1 μm. The positioning algorithm mentioned above was used to calculate the coordinates of the test mark while it moved up and down along with the focusing positioning setup and then the waving trace was characterized. Experimental results indicate that the mechanical swing trace can offer a good reproducibility, if the ascending and descending of the focusing setup are considered,respectively. Based on the experimental results, the model for the swing trace fitting was established and an error correction method was presented. Finally, experiments were performed to test the calibration accuracy of the model.Obtained results demonstrate that the error from the mechanical instability has been reduced to 1.29 μm from 2.84 μm, which can meet the overall alignment requirement of 2 μm.