Holography, which can record and reconstruct all the information of object light waves, has made remarkable progress since its invention. In recent years, the emergence of vectorial holography has brought new developments to this field. Vectorial holography not only inherits the ability to record the amplitude and phase of traditional scalar holography but also introduces the additional control of the polarization dimension, which can significantly improve the density of the recorded information and has been widely used in many fields. This paper discusses vectorial holography in depth from the perspective of polarization modulation. Firstly, the concepts of scalar holography and vectorial holography are introduced, and their advantages and disadvantages are compared. Then, the two polarization modulation methods of vectorial holography are introduced in detail, including the polarization modulation of incident light and output light. Meanwhile, the applications of vectorial holography in the field of 3D display and encryption are described. Finally, the challenges faced by vectorial holography are summarized, and the future development of vectorial holography is expected.