Remote sensing image segmentation is a crucial application in the field of remote sensing image processing. A semantic segmentation network MAE-Net combining multiscale attention and edge supervision is proposed based on the deep learning network U-Net to address the phenomena of building missing classification, missing segmentation, and inaccurate building contour segmentation in the building segmentation of remote sensing images via convolution neural network. First, a multiscale attention module is introduced into each layer during the coding stage. The module separates the input feature map into equal channels and employs the convolution kernels of various sizes for feature extraction in each group. Thereafter, the channel attention mechanism is used in each group to gain more efficient features through self-learning to solve the problem of inaccurate feature extraction of buildings of various sizes. Second, in the decoding stage, the edge extraction module is introduced to build the edge supervision network. The error between the learning edge label and expected edge is supervised by the loss function to aid the segmentation network in better learning the building edge features and make the building boundary's segmentation result more continuous and smoother. The experimental findings show that MAE-Net can completely segment buildings from remote sensing images with complicated and diverse backgrounds and large-scale changes, and the segmentation accuracy is higher.