Aiming at the problem that the face recognition algorithm based on double space local directional pattern (DSLDP) only uses differential operation to extract features, we present a novel approach based on the double operation local directional pattern (DOLDP) for face recognition. Firstly, 3 pixel×3 pixel neighborhood of facial image are convolved with eight Kirsch template operators to obtain eight directions of edge response values. Then, the neighboring edge response values are countered and summed in counterclockwise directions to obtain two sets of eight-direction edge response differences and sums, and the two sets of values are taken as absolute values. Finally, the directions of the maximum values of the two sets of edge response values are encoded into a two-digit octal number to form a DOLDP code. The experimental results on YALE, ORL, AR and CAS-PEAL face databases show that the proposed method combines the sum space and the difference space face feature information, and achieves a better recognition effect. Compared with the intensity space, the sum space face feature information plays a smooth role and shows stronger robustness to light, expression, and occlusion.