Existing single-pixel imaging (SPI) and sensing techniques suffer from poor reconstruction quality and heavy computation cost, limiting their widespread application. To tackle these challenges, we propose a large-scale single-pixel imaging and sensing (SPIS) technique that enables high-quality megapixel SPI and highly efficient image-free sensing with a low sampling rate. Specifically, we first scan and sample the entire scene using small-size optimized patterns to obtain information-coupled measurements. Compared with the conventional full-sized patterns, small-sized optimized patterns achieve higher imaging fidelity and sensing accuracy with 1 order of magnitude fewer pattern parameters. Next, the coupled measurements are processed through a transformer-based encoder to extract high-dimensional features, followed by a task-specific plug-and-play decoder for imaging or image-free sensing. Considering that the regions with rich textures and edges are more difficult to reconstruct, we use an uncertainty-driven self-adaptive loss function to reinforce the network’s attention to these regions, thereby improving the imaging and sensing performance. Extensive experiments demonstrate that the reported technique achieves 24.13 dB megapixel SPI at a sampling rate of 3% within 1 s. In terms of sensing, it outperforms existing methods by 12% on image-free segmentation accuracy and achieves state-of-the-art image-free object detection accuracy with an order of magnitude less data bandwidth.