In the present paper, two metal oxide films of HfO2 and Ta2O5 were prepared by ion beam sputtering technology. Through measuring ellipsometric parameters of HfO2 and Ta2O5 films, their optical constants can be inversion-calculated by nonlinear least squares techniques. In the fitting process, eight experiment groups were arranged by the orthogonal table L8(27). After selecting Cauchy model, the largest influencing factor for fitting optical constant of HfO2 is surface layer model and the largest influencing factor for fitting optical constant of Ta2O5 is refractive index gradient model. The impact of different physical model on MSE and the order for selecting model in the fitting process are determined. According to the selecting model and the determined fitting order, optical constants of the two metal oxide films were inversion-calculated with adding weak absorption model, and the obtained MSE can descend 79% and 39% according to the initial value. The results indicated that the selecting model possesses definite physical significance in the fitting process. The obtained method can be applied in inversion-calculating many metal oxide films with weak absorption. It has wide application value. At 500 nm, the refractive index gradient of Ta2O5 films is greater than HfO2 films, while the extinction coefficient of HfO2 films is greater than Ta2O5. It was shown that Hf metal is easier to form stable oxide than Ta metal. And the absorption of HfO2 films is larger than Ta2O5 films.