We propose pattern self-referenced single-pixel common-path holography (PSSCH), which can be realized using either the digital-micromirror-device (DMD) based off-axis scheme or the DMD-based phase-shifting approach, sharing the same experimental setup, to do wavefront reconstructions. In this method, each modulation pattern is elaborately encoded to be utilized to not only sample the target wavefront but also to dynamically introduce the reference light for single-pixel common-path holographic detection. As such, it does not need to intentionally introduce a static reference light, resulting in it making full use of the pixel resolution of the modulation patterns and suppressing dynamically varying noises. Experimental demonstrations show that the proposed method can not only obtain a larger field of view than the peripheral-referenced approach but also achieve a higher imaging resolution than the checkerboard-referenced approach. The phase-shifting-based PSSCH performs better than the off-axis-based PSSCH on imaging fidelity, while the imaging speed of the latter is several times faster. Further, we demonstrate our method to do wavefront imaging of a biological sample as well as to do phase detection of a physical lens. The experimental results suggest its effectiveness in applications.