We proposed a fiber optic high temperature sensor based on the Mach-Zehnder interference (MZI) structure, which is composed of two lengths of multi-mode fibers (MMFs), a length of few-mode fiber (FMF) and two sections of single-mode fibers (SMFs). Firstly, the two sections of MMFs were spliced with two sections of SMFs. Then, the MMFs were fused to two ends of FMF to form a symmetrically structured fiber-optic MZI structure. In this structure, the MMF served as the optical mode field coupling element, and the cladding and core of the FMF are the interference arm and the reference arm of the MZI structure, respectively. We investigated the sensor’s response characteristics of the temperature and strain. The experimental results indicate that the sensor is sensitive to temperature variation, and the temperature response sensitivity is up to 61.4 pm/℃ in the range of 40—250 ℃, while the sensor has weak strain sensitivity, its strain sensitivity is only −0.72 pm/με in the strain range of 0—1 400 με. Moreover, the sensor has good stability and repeatability. In brief, the proposed fiber optic high temperature sensor has good properties, such as high sensitivity, compact structure, good stability and repeatability, which can be used for monitoring the temperature of submerged oil electric pump units under oil wells.