We propose a high-speed all-optic dual-modal system that integrates spectral-domain optical coherence tomography and photoacoustic microscopy (PAM). A 3 × 3 coupler-based interferometer is used to remotely detect the surface vibration caused by photoacoustic (PA) waves. Three outputs of the interferometer are acquired simultaneously with a multi-channel data acquisition card. One channel data with the highest PA signal detection sensitivity is selected for sensitivity compensation. Experiment on the phantom demonstrates that the proposed method can successfully compensate for the loss of intensity caused by sensitivity variation. The imaging speed of the PAM is improved compared to our previous system. The total time to image a sample with 256-256 pixels is -20 s. Using the proposed system, the microvasculature in the mouse auricle is visualized and the blood flow state is accessed.