A novel fiber-optic toluene sensor is fabricated by using a side-polished fiber (SPF) and reduced graphene oxide (rGO). The sensor can work at room temperature, of which the response time and the recovery time are about 256 s, and its toluene concentration lowest detection limit is below or equal to 7.9×10-5. The sensor possesses a good linearity and reversibility during the low toluene concentration detection. The estimation based on the theoretical model of sensing mechanism indicates that the doping level of graphene has an important influence on the sensitivity of the sensor, and the sensitivity has a remarkable promotion through adjusting the chemical potential of graphene. The influence of retained water molecules on the sensing process can be evaluated or even utilized by monitoring or controlling the humidity. This sensor possesses a large potential to realize low-power, low-cost, fast, sensitive and highly reversible toluene detection.