Cryogenic imaging spectrometer with wide spectrum and high resolution can be used to detect interplanetary mineral component and atmospheric composition. Due to the temperature of detected target in space exploration is very low, the infrared spectra in imaging spectrometer has to work in cryogenic environment to limit the noise from the imaging spectrometer itself to promote its spectral resolution. That is a challenge for the design of cooling system. Based on survey results about the cooling system of VIRTIS for Venus and Comet, the cooling system for cryogenic imaging spectrometer with wide spectrum and high resolution was designed and simulated. The simulation results show the temperature gradient of M and H detector is only 4×10^-2 K. Detector temperature is less than 70 K. The shell temperature is 130±1 K and frame temperature is 200±1 K. All the results demonstrate the designed cooling system meets the design requirements. The results are significant for cooling system design in cryogenic imaging spectrometer, which is used widely in space exploration.