The detection performance of Top-Hat is limited by a fixed single structural element, resulting in poor suppression for complex background. This paper proposes two improved Top-Hat algorithms with a progressive relationship. First, the Top-Hat transform is enhanced according to the gray value difference between small targets and their neighborhoods, and a Top-Hat algorithm with two structural elements is demonstrated. The structural elements are designed for dilation and erosion operations, and the operation sequence of the open operation is adjusted to get better the detection performance for small infrared targets. Based on the upgraded method, a Top-Hat infrared small target detection method with adaptive dual structure based on local contrast is present. The prior information can be obtained, and the size of the dual structure elements can be adaptively changed by calculating the local contrast to obtain the saliency map. The gray value difference between the target region and its neighborhood is used to suppress the background and enhance the target. The results show that the proposed adaptive Top-Hat method based on local contrast performs best in the five evaluation indexes compared with similar and non-similar methods.