For fabric defect identification in the textile industry, a three-dimensional (3D) color phase shift profilometry (CPSP) method is proposed. The detecting system is mainly composed of one CCD camera and one digital-light-processing (DLP) projector. Before detection, the system should be calibrated to make sure the camera parameters. The CPSP color grating is projected to the measured fabric by DLP projector, and then it is collected by CCD camera to obtain the grating phase. The 3D measurement can be completed by the grating phase difference. In image acquisition, only invariable grating is projected to the object. In order to eliminate the interference from background light during the image acquisition, the brightness correction method is researched for improving the detection accuracy. The experimental results show that the false rate of detecting the fabric defects is 5.78%, the correct rates of detecting the fabric defects of hole and qualified fabric are both 100%, and the correct rates of detecting the fabric defect of scratch and fold are 98% and 96%, respectively. The experiment proves that the proposed method can accurately identify fabric defects.