Under the complicated sea and sky background, the existing infrared small target detection algorithms have the problem of high false alarm rate. In this paper, the feature differences between the target and the background were deeply analyzed. Firstly, a method based on gray difference and gradient direction consistency was proposed. The small target was enhanced and some background clutter was suppressed. Secondly, the sharp edge background was further suppressed by combining the eigendecomposition method. Finally, the adaptive threshold was used to separate the small target. The experimental results show that compared with the five existing algorithms, the proposed detection algorithm can effectively reduce the false alarm rate in different complex scenes, greatly improve the signal-to-clutter ratio (SCR) and the background inhibitory factor (BSF), and have good robustness.