Vehicle recognition has a large amount of computation and complex extracted features, while the traditional neural network has incomplete features defined by end-layer features. Therefore, we propose a new vehicle recognition method based on multi-layer features of the convolutional neural network (CNN) and support vector machine (SVM). Firstly, the CNN model is constructed based on the traditional AlexNet model, while the optimal vehicle recognition model is obtained by analyzing the effect of parameter change on the accuracy. Further, the multi-layer vehicle feature map is extracted, and a multi-attribute vehicle feature vector is formed by the serial fusion method and the principal component analysis to enhance the comprehensiveness of the feature and reduce the computational complexity. Finally, vehicle recognition is realized by using the SVM classifier instead of the output layer of CNN, which improves the generalization and error-correction abilities of the model. The experimental results reveal that the proposed method has remarkable performance in classification accuracy and recognition speed; additionally, it has better robustness, compared with the traditional methods.