There boundary between colorectal polyps and normal tissues is not typically evident. Therefore， accurately locating polyp positions is challenging. This study developed a novel polyp image segmentation method based on a combination of multiscale ResNeSt-50 aggregation network and sequential tree-reweighted message passing （TRW-S）. First， a multiscale ResNeSt-50 aggregation network with an encoding–decoding structure was constructed to improve the expressiveness of the network. The encoder of the network is cascaded by convolution module and four-level ResNeSt module to build the ResNeSt-50 backbone network， which realizes linear integration and communication between cross-channel information， ResNeSt-50 uses split attention to strengthen the performance of important channel groups and enhance the ability of the residual module to extract polyp image information. In the bottom three layers of the decoder， a multilayer receptive field block （RFB） was used to obtain multiscale information. Subsequently， the dense aggregation module was used to integrate the output. The decoding information was output by using a fast decoding method， which ensured consistent segmentation performance and reduced the number of parameters. Second， the test-time augmentation （TTA） module was used to improve the prediction accuracy and enhance the generalization ability of the network when generating predictive images. Finally， a sequential tree-reweighted message passing （TRW-S） algorithm based on Markov random fields was constructed to postprocess the predicted image output of the model. This helped achieve continuity of the segmentation edge and consistency within the segmentation region. The experimental results on Kvasir-SEG， an open-access dataset for gastrointestinal polyps images， show that our method achieved an mDice value of 91.6%， mIoU of 86.3%， Smeasure of 92.1%， and MAE of 2.3%，which are higher than those of the polyp segmentation algorithms based on U-NET， U-Net++， ResUNet， SFA， and PraNet. Test results on the unknown datasets ETIS-LaribPolypDB and ColonDB indicate that the proposed model affords improvements in the PraNet and mDice values by 16.4% and 7.7%， respectively. As regards the segmentation performance on the ETIS-LaribPolypDB dataset， the proposed model was found to be highly sensitive to small lesions. Thus， the proposed model exhibits excellent performance in terms of consistency of segmentation area， continuity of segmentation edge， sharpness of contour， and ability to capture small lesions. In addition， it exhibits good generalization ability in the case of unknown datasets.