The shortage for red-light of current commercial white-light LED due to a single yellow-light emitting of YAG∶Ce3+ phosphor restricts its development and application. Ce3+, Ca2+, Gd3+ doped respectively YAG Nano-phosphors in the paper were obtained by using sol-gel method. The phase composition, structure, morphology, particle size, luminescence properties as well as internal quantum efficiency and mechanism of the phosphors with different ions doping content were thoroughly investigated. The particle size of phosphors was 100～200 nm. Pure YAG was obtained as doping Ce3+ and Gd3+, however, the crystal structures expand and X-ray diffractive peaks shifted toward small angles. The crystallinity of samples had no obvious variation with increasing Ce3+ and Ca2+(<0.025 mol) contents, but decreased with increasing Gd3+. The crystal field energy and degree of crystal field split for 5d state of Ce3+ increased with increasing all ions doping contents, and excitation, emission spectra of Gd series phosphors had red shift, whereas Ce, Ca series were negligible due to small doping contents. The photoluminescence (PL) intensity first increased and then decreased with increasing Ce3+ doping contents, and the optimize content was 0.06 mol. However, the PL intensity decreased gradually with the increase of Gd3+ doping contents, but decreased dramatically as increasing Ca2+ contents. The crystal structure was destroyed and PL intensity was close to zero as Ca2+ content run up to 0.03 mol, and impure phases of YAM and YAP produced. The research will provide a theoretical basis and practical reference for the further development and application of nano-YAG phosphors and related functional materials.