For hyperspectral image classification tasks, a graph convolutional network can model the structural and similarity relationships between pixels or regions. To solve the problem of inaccurate construction of an adjacency matrix by calculating the node similarity using the original spectral features of pixels, a graph convolutional network based on spatial-spectral aggregation features (S²AF-GCN) is proposed for feature extraction and pixel-level classification. The S²AF-GCN considers the spatial position of the pixel as the center, aggregates other pixel features in the spatial neighborhood of the pixel, and uses the aggregated pixel features to dynamically update the weights of other pixels in the neighborhood. Through multiple aggregations, the pixel features in the region are smoothed, and the effective feature representation of the pixels is obtained. Next, the aggregated features are used to calculate the similarity and construct a more accurate adjacency matrix. Moreover, the aggregated features are simultaneously used to train the S²AF-GCN to obtain better classification results. The S²AF-GCN achieves overall classification accuracies of 85.51%, 96.95%, and 94.92% on three commonly used hyperspectral datasets, namely, Indian Pines, Pavia University, and Kennedy Space Center, respectively, using 1% labeled samples.