To overcome the difficulty of obtaining large annotated datasets, a proxy task based on a diffusion model was introduced, allowing for self-supervised learning of a priori knowledge from unlabeled datasets, followed by fine-tuning on a small labeled dataset. Inspired by the diffusion model, different levels of noise are weighted with the original images as inputs to the model. By training the model to predict the input noise, a more robust learning of the representation of intravascular ultrasound (IVUS) images at the pixel level was achieved. Additionally, the combined loss function of mean square error (MSE) and structural similarity index (SSIM) was introduced to improve the performance of the model. The experimental results of this method on 20% dataset demonstrate that the Jaccard metric coefficients of the lumen and meida are increased by 0.044 and 0.101, respectively, compared with result of random initialization, and the Hausdorff distance coefficients are improved by 0.216 and 0.107, respectively, compared with result of random initialization, which is similar to the result of using 100% dataset for training. This framework applies to any structural image segmentation model and significantly reduces the reliance on ground truth while ensuring segmentation effectiveness.