Photonic integrated switches that are both space and wavelength selective are a highly promising technology for data-intensive applications as they benefit from multi-dimensional manipulation of optical signals. However, scaling these switches normally poses stringent challenges such as increased fabrication complexity and control difficulties, due to the growing number of switching elements. In this work, we propose a new type of dilated crosspoint topology, which efficiently handles both space and wavelength selective switching, while reducing the required switching element count by an order of magnitude compared to reported designs. To the best of our knowledge, our design requires the fewest switching elements for an equivalent routing paths number and it fully cancels the first-order in-band crosstalk. We demonstrate such an ultra-compact space-and-wavelength selective switch (SWSS) at a scale of $4\times 4\times 4\lambda$ on the silicon-on-insulator (SOI) platform. Experimental results reveal that the switch achieves an insertion loss ranging from 2.3 dB to 8.6 dB and crosstalk levels in between $-35.3\mathrm{dB}$ and $-59.7\mathrm{dB}$. The add-drop microring-resonators (MRRs) are equipped with micro-heaters, exhibiting a rise and fall time of 46 μs and 0.33 μs, respectively. These performance characteristics highlight the switch’s ultra-low element count and crosstalk with low insertion loss, making it a promising candidate for advanced data center applications.