An Yb-doped all-fiber Q-switched laser based on fiber saturable absorber was reported. In order to obtain laser pulses output with a higher peak power and a narrower pulse width, the saturable absorption effect of Ytterbium-doped fiber was used with laser. The 20/130 μm large mode double cladding Yb-doped fiber was used as the gain fiber. The 10/130 μm single mode double cladding Yb-doped fiber was used as saturable absorber. The laser adopted a all fiber-optic structure, and a relatively stable pulse output was obtained at a relatively low cost. The maximum average power was 3 W, the slope efficiency was 30%, the repetition rate was 10–100 kHz, and the minimum pulse duration was 344 ns with the spectral width of 0.05 nm and the central wavelength of 1 064 nm.