The advancement of terahertz technology in recent years and its applications in various fields lead to an urgent need for functional terahertz components, among which a terahertz switch is one example of the most importance because it provides an effective interface between terahertz signals and information in another physical quantity. To date many types of terahertz switches have been investigated mainly in the form of metamaterials made from metallic structures and optically-active medium. However, these reported terahertz switches usually suffer from an inferior performance, e.g., requiring a high pump laser power density due to a low quality factor of the metallic metamaterial resonances. In this paper, we report and numerically investigate a symmetry-broken silicon disk based terahertz resonator array which exhibits one resonance with ultrahigh quality factor for normal incidence of the terahertz radiations. This resonance, which can never be excited for regular circular Si disks, can help to realize a superior terahertz switch with which only an ultra-low optical pump power density is required to modify the free carrier concentration in Si and its refractive index in the terahertz band. Our findings demonstrate that to realize a high terahertz transmittance change from 0 to above 50%, the required optical pump power density is more than 3 orders of magnitude smaller than that required for a split-ring resonator (SRR) based terahertz switch reported in the literature.