In order to improve the apple grading technology, a fruit classification algorithm that coalesce the color model and the apple radius was presented. By using the adaptive threshold method, the binary image can be obtained after graying the sample image. Then, the pixel radii of apple fruits were calculated to get the actual ones based on the relationship between the fruit pixel radius and the true radii. Meanwhile, the H component, containing the color information, was extracted and the red proportion was calculated by converting the RGB image of sample images to HSV space. Finally, the apples can be classified via apple radius and the proportion of red. Compared with the manual classification results, the simulation results show that the algorithm is effective for apple grading, and has higher classification rate up to 96%.