To increase manufacturing efficiency and product quality, robotic arms can replace manual grabbing and other duties. Currently, data-driven algorithms need a significant number of samples to train models, which is less migratory, as well as a specific quantity of processing resources. To address the above issues, this study establishes an efficient robotic arm vision grasping algorithm and system based on RGB-D images. First, we propose a minimum bounding rectangle (MinBRect) target detection algorithm to quickly estimate the target position. Further, the MinBRect is used to calculate the minor axis inclination of the rectangle to estimate the pose for the grasping task, and finally, the UR5 robotic arm is manipulated to perform the actual grasping experiment. In the experiments, the accuracy of the positional estimation of the proposed algorithm for all 10 target objects is above 85.7%, and the average time reaches 0.7677 s. The grasping accuracy and speed of the proposed approach are greatly improved when compared to the two location estimation techniques, indicating that the proposed algorithm has high accuracy and resilience. Furthermore, because no dataset is required, the suggested technique may be used in structured situations with limited computational resources and industrial production, and it has been demonstrated to be portable in power equipment testing. Actual grasping tests are used to validate the effectiveness of the suggested grasping algorithm for the robotic arm system.