Convolutional neural networks (CNNs) have achieved impressive results in hyperspectral image classification. However, because of the limitations of convolution operations, CNNs cannot satisfactorily perform contextual information interaction. In this study, we use the Transformer for hyperspectral classification to address the problem of capturing hyperspectral sequence relationships at extended distances. We propose a multiscale mixed spectral attention model based on Swin Transformer (SMSaNet). The spectral features are modeled using the multiscale spectral enhancement residual fusion module and the spectral attention module in SMSaNet. The spatial features are then extracted using the improved Swin Transformer module, and hyperspectral image classification is realized using a fully connected layer. SMSaNet is compared with five other classification models on two public datasets, that is, the Indian Pines and University of Pavia. The results show that SMSaNet achieves the best classification effect compared to the other models. The overall classification accuracies reach 99.51% and 99.56%, respectively.