As an important branch of machine learning, deep learning has reached another climax of machine learning since its inception. Deep learning has excellent performance in many fields such as image recognition and classification, semantic segmentation, and intelligent driving and so on. At the same time, deep learning algorithms are widely used in the field of optics such as computational hologram generation and imaging, non-parameter reconstruction of digital holography, and spectral resonance curves prediction due to their abstract feature recognition and extraction characteristics, strong model building and generalization capabilities. This article detailed the basic principles of deep learning and its typical application research in image classification, super-resolution imaging, computer generated hologram and digital holography, prediction of surface plasmonics resonance curves, and structural design of metasurfaces. And future development of deep learning in the physical optical field was worth exploring.