This paper reports the experimental realization of efficiently sorting vector beams by polarization topological charge (PTC). The PTC of a vector beam can be defined as the repetition number of polarization state change along the azimuthal axis, while its sign stands for the rotating direction of the polarization. Here, a couple of liquid crystal Pancharatnam–Berry optical elements (PBOEs) have been used to introduce conjugated spatial phase modulations for two orthogonal circular polarization states. Applying these PBOEs in a 4-f optical system, our experiments show the setup can work for PTC sorting with a separation efficiency of more than 58%. This work provides an effective way to decode information from different PTCs, which may be interesting in many fields, especially in optical communication.