In recent years, the classification of hyperspectral images (HSI) based on deep learning has attracted extensive attention in various fields. The number of HSI spectral bands is large and information redundancy is high, which increases calculation complexity. However, a lack of training samples can easily lead to overfitting of model training and reduce classification accuracy. To improve the classification accuracy and reduce the training time, this study proposes a fast dense connection network based on a three-dimensional convolutional neural network (3D-CNN) with double branch and double attention mechanism for HSI classification. In this study, the original data is dimensionless using principal component analysis (PCA) to reduce redundant information. Then, a double efficient channel attention (ECA) mechanism with a double branch dense connection structure and a fast Fourier transform (FFT) is used. It not only ensures the model's classification accuracy but also increases the model's training speed. We conducted experiments on multiple public hyperspectral datasets, and the performance evaluation indices are overall classification accuracy (OA), average classification accuracy (AA), and Kappa coefficient. The experimental results show that the proposed method can improve the classification accuracy and significantly reduce the training time and testing time.