In order to detect objects more accurately in images, an object detection algorithm based on hard example mining and residual network is proposed, which takes faster regional convolutional neural network (Faster R-CNN) as a benchmark. The working principle of Faster R-CNN is described based on deep learning, and the shortcomings and improvement methods of the algorithm are analyzed. Specifically, a deeper residual network is adopted to replace the original ZF or VGG network to extract more effective deep convolution features. In order to enhance the generalization ability of the learning network model, the network parameters are updated with hard examples during training. The experimental results on Pascal VOC2007, Pascal VOC2007+Pascal VOC2012 and BIT show that compared with Faster R-CNN, the proposed method improves detection accuracy by 3.5%, 7.1%, 6.4%, respectively, on the above three datasets.