To accomplish long-range visible and medium-wave infrared whisking broom imaging detection under strict space limitation, dual-band catadioptric shrink-beam system, double fast steering mirrors, and subsequet compact single-wave lenses was used to build a compact dual-band whisking broom imaging system through lens system design optimization. Among them, dual-band catadioptric shrink-beam system was composed of two-mirror Ritchey Chretien system, CaF₂ dichroic prism and subsequet single-wave lenses. The image quality of the shrink-beam system was closed to diffraction limit in the 0.6-0.9 μm and 3.6-4.9 μm wave bands. Image motion of the dual-band shrink-beam system were controlled within halves of the respective pixels during broom imaging process. The effective focal length of the dual-band catadioptric system in the visible band was 1752 mm, there was no lens in between the RC, the three dimensional size of the optical system was 380 mm (axial)×Φ360, the telephoto ratio was 0.22, the line obscuration ratio was 0.34. Based on simulation and analysis, when the incident angles were larger than 30°, the point source transmittance (PST) of the dual-band system was less than 1×10^-4 without additional front baffles. And this system was designed with mature optical cold working, installation and adjustment process and low cost.