The detection of horizon plays a significant role on the system of infrared dim small target detection on images with complex scenes. The deep learning semantic segmentation method is proposed to utilize in horizon detection task. Different from the traditional threshold segmentation method based on manual template, deep learning method digs the spatial information and semantic information of infrared images and has better adaptability. A new method is proposed to utilize the Deeplab-v3+ as the semantic segmentation arithmetic to distinguish sky and ground. The experimental results of scenes with city, forest and mountain show that the proposed method can effectively detect the horizon and demonstrate the superiority robustness of the proposed approach.