In near-shore scenes, under the influence of background, the probability of false detection and low detection accuracy of ship critical parts are high. To address the above problems, a detection network of ship critical parts based on semantic features was proposed, which named critical part detection network (CPDNet). Firstly, by optimizing the network structure and introducing the attention mechanism, the feature expression ability and the perception ability of the ship critical parts were improved. Secondly, based on semantic information, a semantic mask module was designed to reduce the impact of background on detection accuracy. In addition, the angle parameter was added to make the network applicable to targets with orientation. Finally, a ship critical parts dataset, named CP-Ship, was constructed to verify the effectiveness of the proposed network. The experimental results on the CP-Ship dataset show that the average accuracy of the proposed network is 11.35% higher than that of RetinaNet. Compared with other network models, the proposed network performs well in both detection accuracy and speed.