The elastic modulus of biological tissues and materials is the key factor to determine their mechanical properties, which is very important for biomedicine and material monitoring. A method for measuring elastic modulus based on virtual image phase array was proposed. Combining virtual image phase array with confocal microscopy system, the Brillouin spectrum frequency shift of biological tissues and materials could be obtained quickly, and then the elastic modulus could be calculated. Firstly, the relationship between the elastic modulus and Brillouin spectrum frequency shift was analyzed theoretically; Then, the principle of the virtual image phase array and the components of the system were introduced; Finally, the spectrum test of the sample material was performed, and the spectra of polyethylene, silica glass, and polymethyl methacrylate (PMMA) were measured, the longitudinal elastic modulus M was calculated to be 7.23 GPa, 83.4 GPa and 8.92 GPa, respectively, and the Brillouin spectral image of PMMA was measured. The results show that the cascade use of the virtual image phase array and the diffraction grating can improve the order overlap phenomenon of the etalon, and increase the resolution and acquisition rate of the Brillouin spectrum. The laser and backscattering light suppression module designed by this system can significantly improve the contrast of the Brillouin signal which is of great significance in material properties, structure monitoring and biomedicine.