With the rapidly increasing bandwidth requirements of optical communication networks, compact and low-cost large-scale optical switches become necessary. Silicon photonics is a promising technology due to its small footprint, cost competitiveness, and high bandwidth density. In this paper, we demonstrate a $12\times 12$ silicon wavelength routing switch employing cascaded arrayed waveguide gratings (AWGs) connected by a silicon waveguide interconnection network on a single chip. We optimize the connecting strategy of the crossing structure to reduce the switch’s footprint. We develop an algorithm based on minimum standard deviation to minimize the port-to-port insertion loss (IL) fluctuation of the switch globally. The simulated port-to-port IL fluctuation decreases by about 3 dB compared with that of the conventional one. The average measured port-to-port IL is 13.03 dB, with a standard deviation of 0.78 dB and a fluctuation of 2.39 dB. The device can be used for wide applications in core networks and data centers.