Macadamia nut is easy to spoil after being stripped off because of the high level of oil in it. Most of the existing traditional methods are destructive which are difficult to satisfy the demand of non-destructive detection. As one of the widely used deep learning models, convolutional neural network (CNN) has stronger capabilities of feature extraction and model formulation than shallow learning methods and great potential for the application of spectral data. We studied suitable CNN architecture to extract spectral features of Macadamia based on Vis-NIRS analysis, and proposed an efficient non-destructive method to identify the quality of Macadamia. At first, we took three kinds of macadamia nut with different qualities (including better nut, worse nut and moldy nut) as the research object and analyzed the spectral information in the wavelength range of 500~2 100 nm. We introduced the concept of whitening in data preprocessing to strengthen the correlation difference. In the process of model training, we divided the sample into training set and prediction set randomly and then discussed the effects of different structure parameters, such as the number of convolution layer, size of convolution kernel, pooling type, number of neuron in full connection layer and activation function. We applied ReLU and Dropout to prevent over-fitting caused by lack of data. At last, through the analysis of the classification accuracy and computational efficiency, a CNN model of 6-layer structure was established: input layer-convolution layer-pooling layer-full connection layer（including 200 neurons）-full connection layer（including 100 neurons）-output layer. The results show that the final classification accuracy of the calibration set and prediction set reached 100%. This improved CNN model can fully learn the spectral features of macadamia and classify effectively. The combination of the deep learning theory and the spectral analysis method can identify the quality of macadamia accurately, and provide a new idea for the efficient, non-destructive, real-time, online detection of macadamia and other nuts.