To address the problems of a small infrared imaging area, low resolution, and ease of occlusion—resulting in incorrect detection, missed detection, and low detection accuracy—this paper proposes an infrared small-target detection algorithm based on SSE-YOLO. Firstly, a depth non-stepwise convolution module is introduced on the basis of YOLOv8s to avoid the loss of fine-grained information during the detection process and to improve the efficiency of feature learning. Then, a detection layer specialized for small targets is added in the feature extraction stage to improve the model's ability to extract infrared small targets. In addition, an efficient dual attention mechanism (EDAM) is designed to adaptively learn the importance of each channel and spatial location to better capture key information in the image. Secondly, the Shape_IoU loss function is used to focus on the shape of the boundary itself and its scale, which further improves the accuracy of boundary regression. Finally, a series of experiments were conducted on the FLIR dataset and a dataset captured by IRay. The results show that the average accuracies of the proposed method on the two datasets reach 89.8% and 92.1%, which are 3.3% and 2.9% higher than those of the original model, respectively.