In order to study the damage and failure process of steel fiber reinforced concrete and the mechanism of crack development and evolution, based on fractal theory and extended finite element method, the mesoscopic finite element model of steel fiber reinforced concrete cube tensile test and the finite element model of notched beam three-point bending test were established. The reliability of the established finite element analysis model was compared and verified based on the relevant test results. The damage evolution process of steel fiber reinforced concrete was characterized by the fractal dimension of crack, and the effects of different volume content and length of steel fiber, the shape of coarse aggregate and other important factors on the damage evolution process of steel fiber reinforced concrete were investigated. The results show that the damage value based on the fractal dimension of crack can better reflect the damage evolution process and characteristics of steel fiber reinforced concrete. The increase of steel fiber volume content and length and the irregularity of aggregate shape will delay the damage evolution process of steel fiber reinforced concrete cube specimens. The increase of steel fiber volume content and the distance between the initial crack and the midspan, and the reduction of the initial crack height ratio can delay the damage evolution process of steel fiber reinforced concrete notched beams to a small extent.