So far, many investigations had been made on the concentration and species distribution of heavy metals in aquatic environments. However, there are only a few studies on heavy metals in upper reaches of the Yellow River, especially in Gansu, Ningxia and Inner Mongolia sections. We have literatures related to the Yellow River, in this work, we remarkably discussed about the contents, speciation and potential risks of Cd, Pb, Cr, V, Co, Ni, Cu, and Zn in surface sediments from 12 sampling sites in Gansu, Ningxia, and Inner Mongolia sections of the Yellow River of China in 2011 year wet season by high resolution inductively coupled plasma mass spectrometer (HR-ICP-MS) and sequential extraction procedure of BCR method. The results indicated that the metals contents were arranged as Cr>V>Zn>Cu>Ni>Pb>Co>Cd in all sites. Comparing with the background value of soil in local section, Cd showed the highest level at S5 (1.30 μg·g-1), which was almost 13 times higher than the background value (0.103 μg·g-1). Pollution assessment indicated that Cd presented a strong polluted status with the geo-accumulation index (Igeo) value of 3.08 at S5, moderately to strong polluted status with the Igeo ranged from 2.02 to 2.90 in Inner Mongolia section (S1—S4). Moreover, enrichment factor (EF) showed that all heavy metals in these sediments have been influenced by anthropogenic activities. According to potential ecological risk index (RI), S5 and S3 demonstrated high ecologic risk of heavy metals, while other sampling sites showed moderately ecological risk. The results of BCR exhibited that Cd was the most available metal, followed by Co and Ni, while V and Cr were unavailable in the sediments. Risk assessment code (RAC) exhibited high risk for Cd at S1—S4 and very high risk at S5, while medium risk for Ni and Co at all sites. The results and conclusions may be important information and therefore of interest to the relevant departments of the governments.