In complementary learners for real-time tracking known as Staple, the merging coefficients of histogram of oriented gradient feature and color histogram have both a fixed value of 0.3, which can easily cause the problem of losing target when they are merged under different features. To solve this problem, this study proposes an adaptive merging algorithm of complementary learners for real-time visual tracking based on an object probabilistic model known as amStaple, which uses a piecewise function to obtain the adaptive merging coefficient. Experiments on popular object tracker benchmarks including OTB-2013 and OTB-100 verify the effectiveness of the proposed algorithm. Results show that amStaple has better performance than Staple. Compared with Staple in terms of OTB-2013 and OTB-100, amStaple has 6.52% and 3.32% higher precision and 4.89% and 3.11% higher success rates, respectively. Although the proposed algorithm is relatively less innovative, its performance has been obviously improved in various aspects compared with that of a state-of-the-art algorithm from the same period. However, amStaple performs poorly on partial sequence attributes of object tracker benchmarks. To solve this problem, a decision condition is added based on amStaple, which is called amStaple1.