A new asymmetric optical image encryption algorithm was proposed, which combined vector decomposition and phase-truncated. The plaintext was encrypted by four keys to obtain uniformly distributed ciphertext and three decryption keys. The decryption key was generated in the encryption process, which was different from the encryption key. It realized asymmetric encryption and increased the security of the system. The decryption key generated in the process of vector decomposition was strongly related to plaintext. Compared with the existing optical asymmetric encryption algorithms, plaintext was more sensitive to ciphertext and decryption keys. The system was more resistant to selective plaintext attack. At the same time, it also improved the sensitivity of decryption keys. The introduction of phase-truncated enlarged the key space and enhanced the security of the algorithm. Moreover real number ciphertext was produced for easier transmission. The experimental results show that the algorithm has uniform ciphertext distribution and low correlation between adjacent pixels. The decrypted keys and the plaintext to decrypted keys and ciphertext are highly sensitive. This algorithm has strong ability to resist various attack and better optical image encryption effect.