Metasurfaces have emerged as a flexible platform for shaping the electromagnetic field via the tailoring phase, amplitude, and polarization at will. However, the chromatic aberration inherited from building blocks’ diffractive nature plagues them when used in many practical applications. Current solutions for eliminating chromatic aberration usually rely on searching through many meta-atoms to seek designs that satisfy both phase and phase dispersion preconditions, inevitably leading to intensive design efforts. Moreover, most schemes are commonly valid for incidence with a specific spin state. Here, inspired by the Rayleigh criterion for spot resolution, we present a design principle for broadband achromatic and polarization-insensitive metalenses using two sets of anisotropic nanofins based on phase change material Ge₂Sb₂Se₄Te₁. By limiting the rotation angles of all nanofins to either 0 deg or 90 deg, the metalens with a suitable numerical aperture constructed by this fashion allows for achromatic and polarization-insensitive performance across the wavelength range of 4–5 μm, while maintaining high focusing efficiency and diffraction-limited performance. We also demonstrate the versatility of our approach by successfully implementing the generation of broadband achromatic and polarization-insensitive focusing optical vortex. This work represents a major advance in achromatic metalenses and may find more applications in compact and chip-scale devices.