With the development of observation apparatus, the spectra number rises constantly. How to further improve the classification performance deserves to be research. Because of this, the stellar spectra are taken as the research object, the high-performance hybrid deep learning network is proposed based on integrating the advantages of the BERT model in feature extraction and the CNN model in automatic classification, to verify the effectiveness of improving the spectra classification performance. Firstly, the stellar spectra are input into the the BERT model; And then the part in BERT model named Transformers are used to extract the features and based on which, the feature vectors are formed; Finally, the above feature vectors are input into the CNN model, and the stellar spectra classification results can be obtained with the help of softmax classifier. Python3.7 writes the models used in the experiment and the deep learning framework named TensorFlow is introduced. The K-, F-, G-type stellar spectra in SDSS DR10 are used for the experimental dataset, normalized by the min-max normalization method. The effectiveness of the BERT-CNN model is verified by comparing with the support vector machine models (SVM) and CNN. The performances of the above models are related to the parameters, and therefore, the ten cross-validation and the grid search method are used to obtain the optimal experimental parameters. There are two parts to the experiment. One is to evaluate the classification performances of BERT-CNN with precision P, recall R and F1 values. The proportion from 30% to 70% of the experimental dataset is respectively used for the training dataset, and the remainder is used for the test dataset. P, R and F1 values rise with the training size on the K-, F-, G-type stellar datasets. In the case of the same training size, the values of P, R and F1 arrive at the highest, followed by the performance on the G-type stellar dataset, the classification results on the F-type stellar dataset are much poorer. The other experiment is to evaluate the classification performances of SVM, CNN and BERT-CNN with accuracy. The classification performances of BERT-CNN on the K-, F-, G-type stellar datasets are all best, followed by CNN. The classification accuracies of SVM are much lower than the other two models. It indicates that the BERT-CNN model contributes to improving the spectra classification performance.