Ultra-short pulse lasers at 1.0 μm waveband have important applications in many fields such as laser processing, optical precision measurement, biomedicine etc. However, because the Yb-doped fiber laser operates in the all-normal dispersion regime, the pulse width of the laser output is usually large. Because the microfiber could provide anomalous dispersion at 1.0 μm waveband by changing its size, in order to achieve ultra-short pulse the microfiber was employed as dispersion compensation device to compress the Yb-doped fiber laser pulse. In this work, the diameter and length of taper waist of the home-made microfiber were 3 μm and 5 cm, respectively. The pulse width of the Yb-doped fiber laser was 37.6 ps, while it was compressed to be 8.5 ps by the microfiber. These results offer a more simple and inexpensive method for pulse compression.