Oil spill pollution on the sea surface is one of the most common marine pollutions, which usually exists in the state of non-emulsification, emulsification and other weathering, and the emulsification stage is more harmful to the ocean. Therefore, it is of great significance to quickly monitor the oil spill information and accurately identify and evaluate the emulsified oil spills pollution for the emergency treatment of oil spill and the restoration of the ecological environment. Laser-induced fluorescence (LIF) is recognized as one of the most effective detecting oil spills. LIF detection systems can be divided into the forms of coaxial transceiver and non-coaxial transceiver. Since there is no research on transmitting and receiving related problems in the detection of the emulsified oil spill by LIF system with coaxial transceiver, the optical parameters such as absorption coefficient and scattering coefficient of the emulsified oil spill are calculated by MIE scattering theory, and the Monte Carlo, photon transport model, is established to simulate the bidirectional reflectance and reradiation distribution functions (BRRDF) of the emulsified oil spill. The relationship between f_BRRDFcos²θ and the transmitting and receiving angle of the emulsified oil spill is analyzed under multi parameters of concentration, thickness and oil type. Then the suitable conditions for detecting oil spill on the sea surface based on an LIF system with coaxial transceiver are obtained. The results show that the f_BRRDFcos²θ is independent of the azimuth angle of transmitting and receiving. However, it is greatly affected by the zenith angle of transmitting and receiving, the variation law of f_BRRDFcos²θ of the emulsified oil spill under various parameters has certain differences. The f_BRRDFcos²θ of water-in-oil of heavy oil and oil-in-water of low concentration are more sensitive to the change of zenith angle, and the f_BRRDFcos²θ of water-in-oil of light oil and oil-in-water of high concentration are insensitive to smaller angle (0°~45°), and then decrease rapidly. Therefore, when detecting the emulsified oil spill on the sea surface based on the LIF system with a coaxial transceiver, it is advisable to transmit and receive the zenith angle within the range of 0°~45°, and the maximum optical power can be received at 0°. In addition, to verify the correctness of the simulation, the fluorescence spectra of the emulsified oil spill were measured by the laboratory LIF system. The results show that this is consistent with the simulation results.