The application of UAV in the field of electro-optical reconnaissance is more and more widely. Desining a robust flight controller for UAV is an essential technique. A flight controller design method based on model matching and genetic algorithm optimization was proposed in this paper. A nonlinear model was used as the controlled object of this method. Compared with the classic flight controller design method, this method could access the needed controllers quickly and conveniently. A nonlinear mathematical six-degree-of-freedom model of a UAV was established, the aerodynamics, engine and environmental model were all included in this model. Using the above method, a UAV flight controller was designed. Based the finite state machine theory, the flight management model was established to design flight modes and carry out the controller switch. Finally, the six degrees of freedom nonlinear simulation was carried out to verify the effectiveness of the designed controller.