An appropriate reference spectrum is essential for the direct-sun differential optical absorption spectroscopy (DS-DOAS). It depends on the real reference spectrum to retrieve the total vertical column density (VCD). The spectrum detected at the time with minimum sun zenith angle under the relative clear atmospheric condition in the measurement period was conventionally selected as the reference spectrum. Because there is still untracked NO2 absorption structure in the reference spectrum, the VCD retrieved based on the above spectrum is actually relative VCD, which results in larger error. To solve this problem, a new method was investigated. A convolution of extraterrestrial high-precision solar Fraunhofer spectrum and the instrumental function of the spectrometer was computed and chosen as the reference spectrum. The error induced by NO2 absorption structure in the reference spectrum was removed. Then the fitting error of slant column density (SCD) retrieved by this method was analyzed. The correlation between the absolute SCD and the differential slant column density (dSCD) was calculated. The result shows that the error of SCD retrieved by this new method is below 1.6×1016 molecules·cm-2 on March 7, 2011, while the error generated by the normal method is about 4.25×1016 molecules·cm-2. The new method decreased more than 62% error. In addition, the results throughout the day were compared to the troposphere VCD from MAX-DOAS and they are in good agreement. It indicates that the new method could effectively reduce the VCD error of the common way.