The existing light field image depth estimation technique cannot make a balanced estimation between major object and background. In this paper, a novel depth estimation method based on light field structure characteristic and multiview matching was proposed for light field image. The light field structure guided method was used as the basis of depth estimation. In order to maintain a smooth transition of the depth changing region and consider that the light field image has multiview subaperture image arrays, the multiview matching was presented to optimize the depth estimation. In the Markov random field domain, a smooth term was constructed based on the characteristics of the light field structure and a data term was constructed based on multiview stereo matching. Then, an optimization method utilizing the above two terms was proposed to balance object depth boundary and background depth estimation, so as to improve the depth estimation of light field images. Experimental results show that the proposed method can produce high quality depth estimation results with clear depth boundary and accurate depth in background.