As an important part of an aircraft, aircraft skin is directly related to the occurrence of aircraft accidents, so the detection of aircraft skin damage is of great significance for the effective prevention of aviation safety accidents. In this paper, an improved rotated object detection method based on box boundary-aware vectors (BBAVectors) is proposed. First, for the problem of large variation of damage scale, the feature fusion network (FFN) is used to improve the multi-scale detection effect. Second, for the problem of lots of background noise in aircraft skin images, coordinate attention (CA) mechanism is introduced to enhance the object feature information. Finally, for the problem of arbitrary skin damage distribution direction, the damage position of arbitrary angle is represented by the BBAVectors to improve the accuracy of object localization. The experimental results show that the improved rotated object detection method improves the mean average precision by 5.7 percentage points compared with the original model, and the detection performance is better than the horizontal object detection method. On the basis of effectively solving the effect of aircraft skin damage detection in arbitrary directions, it provides better technical support for the improvement of aircraft damage detection methods.