Subwavelength grating structures exhibit excellent notch filtering properties. The classical design is to find the optimal solution by setting the geometric structure parameters of the subwavelength, solving Maxwell's equations, and setting an optimization algorithm. It consumes a lot of time and computing resources. This paper presents an inverse design method based on deep learning and constructs a series neural network which can realize both forward simulation and inverse design. The Tensorflow library based on Python language is constructed to optimize the height of uniform waveguide layer, the height of sub-wavelength grating, refractive index, period and duty cycle, and to study the characteristics of sub-wavelength grating notch filtering in the range of 0.45~0.7 μm. Using rigorous coupled wave analysis (RCWA) numerical simulation to generate 23 100 data sets, 18 480 data sets were randomly selected as training sets, and 4 620 data sets were used as test sets, the network node and Batch were studied. The results show that the network performance is best when the network model structure is 5×50×200×500×200×26, and the Batch size is 128 after 1 000 iterations. Compared with the independent network model, the loss rate of the forward simulation test set of the series neural network decreased from 0.033 63 to 0.004 5, and that of the reverse design decreased from 0.702 98 to 0.052 98. At the same time, the problem that the network can not converge in the reverse design process caused by the non-uniqueness of data is solved. The geometric structure parameters of the sub-wavelength grating are given in 1.35 s on average by inputting any spectral curve into the trained network, and the correlation between the parameters and the RCWA numerical simulation curve is analyzed, the similarity coefficients of the curves were all greater than 0.658 1, which belonged to strong correlation. In addition, a red, green and blue notch filter is designed, whose peak reflectivity can reach 98.91%, 99.98% and 99.88% respectively. Compared with the traditional method, this method can quickly and accurately calculate the geometric parameters of the grating. It provides a new method for sub-wavelength grating design.