Multi-spectral video imaging can simultaneously collect spatial, temporal and spectral information of scenes. It can be widely used in many fields such as remote sensing, agricultural monitoring, material analysis, etc. However, traditional spectrometers often depend on optical dispersion spectroscopy structure, which makes the system complex and difficult to calibrate, and it is difficult to popularize. Therefore, a miniaturized multi-sensor spectral video imaging system was built in this paper. A fast alignment method of multi-view image or video based on camera poses was proposed to realize real-time acquisition and alignment of multi-spectral video on embedded system. Through the experimental verification of complex scenes, the alignment method proposed in this paper has achieved desired results in both objective index (e.g. PSNR, SSIM) and subjective visual effects.