To reduce the time consumption of the three-axis vision measurement system in the focusing process and improve the accuracy of focusing, an auto-focus algorithm based on the optical defocus model was proposed. The evaluation function of the auto-focus algorithm adopted the Tenengrad gradient function. The search algorithm was divided into two steps: (1) The optical defocus model was decomposed into two curve functions, the two curve functions were solved through collecting the sharpness values of 4 images and their x-axis coordinates and the intersection position of the two curves was obtained. The intersection position was the approximate position of the on-focus position; (2) One image was collected at the intersection position and two images were collected on the left and right sides of the intersection point. The mean value of the fitting Gaussian function was obtained by Gaussian function fitting, which was the exact on-focus position. To verify the method, firstly, 10 times repeatability experiment was processed, the error of repeatability was 4.1 μm. Secondly, an image at rough focus position was obtained and 2 images on the left of rough focus position and 2 images on the right were obtained. The precise focus position was given by Gaussian fitting. The 10 times repeatability error of precise focus position was 5.1 μm. The algorithm only needed to collect 9 images, and the synthetic standard uncertainty was 2.12 μm. The algorithm improves the focusing efficiency and accuracy of the three-axis vision measurement system.