We theoretically study the temporal contrast degradation by the spectral phase distortion in chirped pulse amplification (CPA) lasers. As an example, we analyze the impact of the surface qualities of the optical elements such as mirrors and grating in the stretcher and compressor on the temporal contrast. The temporal contrast declines fast in the case of a rapidly varying random surface error of the optical elements. When the values of PV, RMS and GRMS of the surface error curve are reduced, the temporal contrast is becoming better and better. And the temporal contrast can be improved after the surface error curve is to be spatial filtering. Those results are helpful for the choice of the surface parameters of the optical elements in the stretcher and compressor.