A fine-grained image recognition method based on the progressive fusion of cross-grained features is proposed to address the problems of small differences between classes of fine-grained images, difficulty in capturing discriminative features and low recognition accuracy. First, the random region confusion module is used to generate images with different granularity levels for training various stages of the backbone network ConvNeXt. Second, the image representation of differing granularity in the middle layer of the model is enhanced using the random sample-swapping module. Then, the progressive multigranularity training strategy and mutual belief channel loss function are used for model training to fuse cross-granular information collaboratively. Finally, to obtain the final recognition results, the multigranularity features are integrated and fused to combine the classifiers. The experimental results demonstrate that the recognition accuracies of this method on three public datasets are 92.8% (CUB-200-2011), 95.5% (Stanford Cars), and 94.0% (FGVC-Aircraft), which are better than the current mainstream fine-grained image recognition methods. The recognition accuracy on the self-constructed Lock-Hole dataset reaches 97.3%, and the average recognition time of a single image is 0.016 s, which can realize the accurate recognition of the lock-hole image and satisfy the requirement of fast lock-hole recognition in emergency unlocking scenarios.