The concentration of MgO is one of the most important parameters in the phosphate ore flotation. The fast detection of the concentration of MgO has great significance for the optimization of the flotation, the improvement of the efficiency and the reduction of the cost. Therefore, LIBS (Laser Induced Breakdown Spectroscopy) is introduced into the analysis of Mg in phosphate. But the common used strong lines of Mg (Mg Ⅱ 279.6 nm, Mg Ⅱ 280.3 nm, Mg Ⅰ 285.2 nm) are resonance lines in LIBS analysis. For the effect of the self-absorption, the spectral intensity of the resonance line is lower than the theoretical value, and this intensity reduction will reduce the accuracy of the analysis. In this work, a method based on an approximate function of Voigt profile was proposed. Firstly, simplified the Voigt profile function with the approximate function; secondly, determined the center of the spectral line and the full width at half maximum (FWHM) under ideal conditions with the low concentration samples; then, determined the wing area for fitting by calculating the slope of the spectral area near the spectral peak; finally, obtained the fitting profile closer to the theoretical one, by fitting the approximate function with the wing area selected above. In the application of the quantitative analysis of Mg in phosphate ore, the internal standard method was used forthe calibration. The fitting spectral areas of Mg Ⅰ 285.2 nm and Si Ⅰ 288.2 nm were chosen as the analytical line intensity and the reference line intensity, respectively. Comparing the internal standard method without fitting, the determination coefficient (R2) with the proposed method was improved from 0.923 to 0.998; the root-mean-square error (RMSE) and the average relative error (ARE) were reduced from 0.96 and 38.65% to 0.16 and 2.79%, respectively. The calibration results proved that with the proposed method, the measurement accuracy can be improved significantly for the application of magnesium analysis in phosphorus ore.