TiO₂-Nb₂O₅ mixed optical film has been deposited on BK-7 substrate by the reactive magnetron sputtering technique. The microstructure,morphology,composition and optical properties have been investigated by X-ray diffraction,scanning electron microscopy,atomic force microscopy,X-ray photoelectron spectroscopy and spectrophotometer. The results show that the TiO₂-Nb₂O₅ mixed film is amorphous and has optically flat and uniform surface. The structure of the film is compact and not column structures or defects can be observed,and the stoichiometric ratio of Nb₂O₅ to TiO₂ is about 1.54.Refractive index and extinction coefficient of the film are determined from the normal incidence transmittance,and refractive index is 2.34 and extinction coefficient is 2.0 × 10^－4 at 550 nm wavelength.These results indicate that the mixed film has high quality for optical applications.