A recognition method of mineral Raman spectrum based on all-optical diffraction neural network was proposed. Firstly, the data structure characteristics of the Raman spectra of minerals were analyzed, the similarities and differences between traditional neural network and optical diffractive neural network were compared and analyzed, and the optical diffractive neural network was constructed according to the preprocessed data. Secondly, the cross entropy loss function and Adam algorithm were used to train the optical diffractive neural network, and the optimized network parameters were obtained. Finally, under the simulation conditions, the effects of different grid-height accuracy on the accuracy of mineral recognition were verified and analyzed, and the network accuracy and accuracy loss corresponding to the different grid-height accuracy was given. The test results on the RRUFF mineral Raman spectrum database show that the recognition accuracy of five kinds of minerals is 94.2%, which proves the feasibility of Raman spectrum recognition using optical diffractive neural network. It provides a reference for the application of optical diffractive neural network; the accuracy of five kinds of minerals under the condition of 6 bit grid-height resolution is 93.6%, which proves that grid height discretization can not only ensure the accuracy of network, but also greatly reduce the difficulty of grating fabrication. It provides theoretical support for grating fabrication.