To address the necessity for rapid identification of pollutants in abrupt water-pollution incidents, a lightweight neural network algorithm suitable for portable devices is proposed. By obtaining surface-enhanced Raman spectroscopy (SERS) data of five common water pollutants and performing preprocessing, a two-dimensional Morlet wavelet transform is applied to separate high- and low-frequency signals, thus enhancing feature representation. To improve the model's feature-extraction capability, a multipooling strategy is introduced, and the efficient channel attention (ECA) mechanism is modified to develop a multipooling attention ECA (MP_ECA) module. This module is integrated with the MobileNetV2 network to construct the MobileNetV2_MP_ECA model for wavelet image classification and recognition. The gradient-weighted class activation mapping (Grad-CAM) technique is utilized to generate heatmaps, which further verifies the effectiveness of wavelet transform in enhancing feature extraction and classification accuracy. Experimental results show that the proposed model achieves a classification accuracy of 97.83%, thus outperforming other attention mechanism models, conventional convolutional neural networks, and common machine-learning methods. Additionally, the model size of proposed model is only 6.11 MB and incurs a floating-point computation of 230.20 MFLOPs, thus rendering it suitable for resource-constrained mobile-device applications. This study provides a novel strategy and approach for efficiently detecting pollutants in real-world abrupt water-pollution scenarios.