During the storage and transportation of rice, mildew easily occurs in a suitable temperature and humidity environment will cause a lot of food waste and huge economic losses, which in turn affects food security. This paper proposed a method for detecting the mildew degree of rice-based on near-infrared spectroscopy image processing technology and neural network. First of all, through the agricultural multi-spectral cameras (Sequoia) and fixed light sources and other equipment, this research has constructed a near-infrared image data acquisition platform for moldy rice. The imaging data of the different mold states (three states: healthy rice, mild mold, and moderate mold) of three varieties of Muxiang, Zaoxiang, and Caidao in Heilongjiang area were acquired. Secondly, taking data samples of rice with different degrees of mildew as the research object, for the 160×160 pixel effective area of the infrared spectrum (NIR) image, applying digital image processing technology combined with spectral image analysis methods to study the various texture characteristics and spectral reflectance frequency characteristics of near infrared spectroscopy (NIR) images, optimizing the spectral characteristics of the mildew state of different rice varieties. The texture features (mean, standard deviation, smoothness, third-order distance, consistency, information entropy, average gradient, fractal dimension) of the near-infrared image are extracted, and the reflection value frequency of the NIR spectrum in the 0.2~0.8 interval when the interval step is 0.1, based on a total of 14-dimensional spectral image characteristic index. At last, based on the feature vector of the NIR image, using the feedforward neural network adaptive inference mechanism, a nonlinear mapping model between the degree of rice mildew and its near-infrared image characteristics was established. The network structure of the model is 14-60-3, and the network output code vector is analyze to the rice mildew grade, realizing the rapid detection method of rice mildew degree. The results show that this paper proposes that the detection model reaches the preset target accuracy of 0.06 when the number of learning times is 28 455, and the correlation coefficient between the extracted rice NIR image features and the model output is 0.85. In the simulation test, the average error between the network output value calculated by the detection model and the expected output value is 0.521 39, the variance is 0.137 82, and the standard deviation of the error is 0.371 23. The accuracy of detecting the degree of mildew of different rice is 93.33%. The research results are a new method for realizing the non-destructive detection of the degree of rice mildew and can provide technical support early and automatic and rapid detection of early mildew during rice storage.