Detecting small objects on highways is a novel research topic. Due to the small pixel of objects on highways, tradi- tional detectors have difficulty in capturing discriminative features. Additionally, the imbalance of feature fusion methods and the inconsistency between classification and regression tasks lead to poor detection performance on highways. In this paper, we propose a balance feature fusion and task-specific encoding network to address these is- sues. Specifically, we design a balance feature pyramid network (FPN) to integrate the importance of each layer of feature maps and construct long-range dependencies among them, thereby making the features more discriminative. In addition, we present task-specific decoupled head, which utilizes task-specific encoding to moderate the imbalance between the classification and regression tasks. As demonstrated by extensive experiments and visualizations, our method obtains outstanding detection performance on small object detection on highways (HSOD) dataset and AI-TOD dataset.