Optical fiber is an indispensable transmission medium in modern communication system and quantum secure communication network. In order to solve the problem that the optical fiber end surface defects cause transmission quality decline or even permanent damage to optical transmission system, a fiber end surface detection model YOLOv5_CS based on YOLOv5 algorithm is proposed. Firstly, ShuffleNetV2, a lightweight network, is used as the main feature extraction network. Deep convolution operation and channel random mixing strategy are used to reduce model capacity and enrich feature information. Secondly, the convolutional block attention module (CBAM) is introduced, and features are enhanced in both spatial and channel dimensions to improve network performance. Finally, the number of convolution kernels in the feature fusion layer is reduced to achieve further model compression. The validity of the proposed method is compared and verified by the optical fiber end data set constructed by data augmentation technology. The results show that compared with the YOLOv5 algorithm, the model capacity of the proposed model is reduced by 80%, the detection speed is increased by 31.1 frame/s, and the mean average precision (mAP) is increased by 1.7%, which can accurately and real-time detect optical fiber end surface defects. This work is aimed at the development of portable intelligent detection device, and can also provide technical support for optical fiber end surface defects detection and related visual sensing industry.