Considering the shortcomings of the existing long distance coherent optical communication receiving system using single tube detectors which receives weak signals with large noise and low SNR, a balanced detector is designed for coherent optical communication system. The balanced detector uses two diodes with TIA respectively and reduces most of the laser′ s relative intensity noise after two hybrid magic Tee′ s coupling and subtraction. The coherent optical system is set up to test the performance of the balanced detector. The local-oscillator and signal laser are mixed and then the eye diagram shows that communication is in good condition. By comparing the spectrum, the results show that the noise of the coherent optical communication system is reduced by using the balance detector compared to the single tube. The signal-to-noise ratio of system is enhanced about 10 dB, which proves the balance detector′s feasibility and superiority for application in the coherent optical communication system.