ing at the problem that using RGB images for vehicle detection are affected by complex conditions such as road shadow, vehicle reflection and insufficient light. The paper proposes a vehicle detection algorithm based on convolutional neural network and combination of RGB and depth images. Two improved models of single-channel RG-D and double-channel RGB-D fusion networks are designed to improve detection speed and accuracy respectively. The algorithm is tested with (Grand Theft Auto) vehicle dataset and compared with other popular algorithms based on RGB images. The results show that compared with Yolo v2 algorithm based on RGB images, detection accuracy and recall rates increase 5.69% and 6.31% respectively by double-channel RGB-D fusion network, and the fastest detection speed of single image reaches 24 ms with single-channel RG-D fusion network. Experiments show that the improved network model based on RGB-D images can achieve real-time detection and effectively improve vehicle detection accuracy.