Oxygen vacancies have a profound effect on the magnetic, electronic, and transport properties of transition metal oxides but little is known about their effect on thermal expansion. Herein we report the effect of oxygen defects on the structure formation and thermal expansion properties of the layered perovskite Ca2RuO4 (CRO). It is shown that the CRO containing excess oxygen crystallizes in a metallic L-CRO phase without structure transition from 100 K to 500 K and displays a normal thermal expansion behavior, whereas those with oxygen vacancies adopt at room temperature an insulating S-CRO phase and exhibit an enormous negative thermal expansion (NTE) from 100 K to about 360 K, from where they undergo a structure transition to a high temperature metallic L-CRO phase. Compared to the L-CRO containing excess oxygen, the S-CRO structure has increasingly large orthorhombic strain and distinctive in-plane distortion upon cooling. The in-plane distortion of the RuO6 octahedra reaches a maximum across 260 K and then relaxes monotonically, providing a structure evidence for the appearance of an antiferromagnetic orbital ordering in the paramagnetic phase and the Ag phonon mode suppression and phase flip across the same temperature found recently. Both the L- and S-CRO display an antiferromagnetic ordering at about 150–110 K, with ferromagnetic ordering components at lower temperature. The NTE in S-CRO is a result of a complex interplay among the spin, orbital, and lattice.