Special materials can fake finger veins to deceive recognition systems, and the large amount of computation is required for finger vein recognition using convolutional neural networks. In this study, a finger vein recognition system with an in vivo detection function and a lightweight convolutional neural network structure was designed. Changes in finger pulse waves were detected using Photo Plethysmo Graphy to determine whether the object was living. The ResNet-18 convolutional neural network was improved using a pruning and channel recovery method, and L1 regularization was combined to improve the feature selection ability of the convolutional neural network. On the basis of improving the accuracy of the algorithm, the network can effectively reduce the consumption of computing resources. The experimental results show that by using the improved pruning and channel recovery optimization structure, the number of parameters decreases by 75.6%, the amount of calculations decreases by 25.6 %, and the iso-error rates obtained in the Shandong University and Hong Kong Polytechnic University finger vein databases are 0.025% and 0.085%, respectively, which are significantly lower than those of RESNET-18 (0.117% and 0.213%).