In the vehicle to infrastructure communication, the millimeter wave beam width is narrow and user equipment mobility is high. Therefore, effective millimeter wave beam selection is a key and challenging task. Aiming at the problem of high overhead in the beam search process, this paper proposes a millimeter-wave beam selection algorithm based on radar and location information. The proposed algorithm trains a neural network structure on the lidar and beam tracking channel datasets using the Top-k classification metric. It also predicts the best beam pair using the Global Context Net (GCNet) model. The simulation results show that the performance of the proposed algorithm is significantly improved in the classification and recognition accuracy. At the same time, it only needs to search 5 beams to approach the performance of the exhaustive search, which greatly reduces the beam search overhead.