Research Article|19 Article(s)
Edge accelerated reconstruction using sensitivity analysis for single-lens computational imaging
Xuquan Wang, Tianyang Feng, Yujie Xing, Ziyu Zhao, Xiong Dun, Zhanshan Wang, and Xinbin Cheng
Computational imaging enables high-quality infrared imaging using simple and compact optical systems. However, the integration of specialized reconstruction algorithms introduces additional latency and increases computational and power demands, which impedes the performance of high-speed, low-power optical applications, such as unmanned aerial vehicle (UAV)-based remote sensing and biomedical imaging. Traditional model compression strategies focus primarily on optimizing network complexity and multiply-accumulate operations (MACs), but they overlook the unique constraints of computational imaging and the specific requirements of edge hardware, rendering them inefficient for computational camera implementation. In this work, we propose an edge-accelerated reconstruction strategy based on end-to-end sensitivity analysis for single-lens infrared computational cameras. Compatibility-based operator reconfiguration, sensitivity-aware pruning, and sensitivity-aware mixed quantization are employed on edge-artificial intelligence (AI) chips to balance inference speed and reconstruction quality. The experimental results show that, compared to the traditional approach without hardware feature guidance, the proposed strategy achieves better performance in both reconstruction quality and speed, with reduced complexity and fewer MACs. Our single-lens computational camera with edge-accelerated reconstruction demonstrates high-quality, video-level imaging capability in field experiments. This work is dedicated to addressing the practical challenge of real-time edge reconstruction, paving the way for lightweight, low-latency computational imaging applications.
Advanced Imaging
  • Publication Date: May. 30, 2025
  • Vol. 2, Issue 3, 031001 (2025)
ClearAIM: dynamic quantitative label-free monitoring of tissue optical clearing|Editors' Pick
Kamil Kalinowski, Anna Chwastowicz, Piotr Arcab, Mikołaj Rogalski, Wiktoria Szymska, Emilia Wdowiak, Julianna Winnik, Piotr Zdańkowski, Michał Józwik, Paweł Matryba, and Maciej Trusiak
Achieving cellular-resolution insights into an organ or whole-body architecture is a cornerstone of modern biology. Recent advancements in tissue clearing techniques have revolutionized the visualization of complex structures, enhancing tissue transparency by mitigating light scattering caused by refractive index mismatches and fast-changing scattering element distribution. However, the field remains constrained by predominantly qualitative assessments of clearing methods, with systematic, quantitative approaches being scarce. Here, we present the ClearAIM method for real-time quantitative monitoring of the tissue clearing process. It leverages a tailored deep learning-based segmentation algorithm with a bespoke frame-to-frame scheme to achieve robust, precise, and automated analysis. Demonstrated using mouse brain slices (0.5 and 1 mm thick) and the CUBIC method, our universal system enables (1) precise quantification of dynamic transparency levels, (2) real-time monitoring of morphological changes via automated analysis, and (3) optimization of clearing timelines to balance increased transparency with structural information preservation. The presented system enables rapid, user-friendly measurements of tissue transparency and shape changes without the need for advanced instrumentation. These features facilitate objective comparisons of the effectiveness of tissue clearing techniques for specific organs, relying on quantifiable values rather than predominantly empirical observations. Our method promotes increased diagnostic values and consistency of the cleared samples, ensuring the repeatability and reproducibility of biomedical tests.
Advanced Imaging
  • Publication Date: Apr. 29, 2025
  • Vol. 2, Issue 2, 021003 (2025)
100 fps single-pixel imaging illuminated by a Fermat spiral fiber laser array
Haolong Jia, Guozhong Lei, Wenhui Wang, Jingqi Liu, Jiaming Xu, Wenda Cui, Wenchang Lai, and Kai Han
Single-pixel imaging (SPI) uses modulated illumination light fields and the corresponding light intensities to reconstruct the image. The imaging speed of SPI is constrained by the refresh rate of the illumination light fields. Fiber laser arrays equipped with high-bandwidth electro-optic phase modulators can generate illumination light fields with a refresh rate exceeding 100 MHz. This capability would improve the imaging speed of SPI. In this study, a Fermat spiral fiber laser array was employed as the illumination light source to achieve high-quality and rapid SPI. Compared to rectangular and hexagonal arrays, the non-periodic configuration of the Fermat spiral mitigates the occurrence of periodic artifacts in reconstructed images, thereby enhancing the imaging quality. A high-speed data synchronous acquisition system was designed to achieve a refresh rate of 20 kHz for the illumination light fields and to synchronize it with the light intensity acquisition. We achieved distinguishable imaging reconstructed by an untrained neural network (UNN) at a sampling ratio of 4.88%. An imaging frame rate of 100 frame/s (fps) was achieved with an image size of 64 pixel×64 pixel. In addition, given the potential of fiber laser arrays for high power output, this SPI system with enhanced speed would facilitate its application in remote sensing.
Advanced Imaging
  • Publication Date: Apr. 09, 2025
  • Vol. 2, Issue 2, 021002 (2025)
Self-supervised PSF-informed deep learning enables real-time deconvolution for optical coherence tomography|On the Cover
Weiyi Zhang, Haoran Zhang, Qi Lan, Chang Liu, Zheng Li, Chengfu Gu, and Jianlong Yang
Deconvolution is a computational technique in imaging to reduce the blurring effects caused by the point spread function (PSF). In the context of optical coherence tomography (OCT) imaging, traditional deconvolution methods are limited by the time costs of iterative algorithms, and supervised learning approaches face challenges due to the difficulty in obtaining paired pre- and post-convolution datasets. Here we introduce a self-supervised deep-learning framework for real-time OCT image deconvolution. The framework combines denoising pre-processing, blind PSF estimation, and sparse deconvolution to enhance the resolution and contrast of OCT imaging, using only noisy B-scans as input. It has been tested under diverse imaging conditions, demonstrating adaptability to various wavebands and scenarios without requiring experimental ground truth or additional data. We also propose a lightweight deep neural network that achieves high efficiency, enabling millisecond-level inference. Our work demonstrates the potential for real-time deconvolution in OCT devices, thereby enhancing diagnostic and inspection capabilities.
Advanced Imaging
  • Publication Date: Mar. 18, 2025
  • Vol. 2, Issue 2, 021001 (2025)
SnapCap: efficient snapshot compressive scene captioning
Jianqiao Sun, Yudi Su, Hao Zhang, Ziheng Cheng, Zequn Zeng, Zhengjue Wang, Chunhui Qu, Bo Chen, and Xin Yuan
Describing a scene in language is a challenging multi-modal task as it requires understanding various and complex scenes, and then transforming them into sentences. Among these scenes, the task of video captioning (VC) has attracted much attention from researchers. For machines, traditional VC follows the “imaging-compression-decoding-and-then-captioning” pipeline, where compression is a pivot for storage and transmission. However, in such a pipeline, some potential shortcomings are inevitable, i.e., information redundancy resulting in low efficiency and information loss during the sampling process for captioning. To address these problems, in this paper, we propose a novel VC pipeline to generate captions directly from the compressed measurement, captured by a snapshot compressive sensing camera, and we dub our model SnapCap. To be more specific, benefiting from signal simulation, we have access to abundant measurement-video-annotation data pairs for our model. Besides, to better extract language-related visual representations from the compressed measurement, we propose to distill knowledge from videos via a pretrained model, contrastive language-image pretraining (CLIP), with plentiful language-vision associations to guide the learning of our SnapCap. To demonstrate the effectiveness of SnapCap, we conduct experiments on three widely used VC datasets. Both the qualitative and quantitative results verify the superiority of our pipeline over conventional VC pipelines.
Advanced Imaging
  • Publication Date: Feb. 25, 2025
  • Vol. 2, Issue 1, 011003 (2025)
Coded self-referencing wavefront shaping for fast dynamic scattering control|Editors' Pick
Zhengyang Wang, Daixuan Wu, Yuecheng Shen, Jiawei Luo, Jiajun Liang, Jiaming Liang, Zhiling Zhang, Dalong Qi, Yunhua Yao, Lianzhong Deng, Zhenrong Sun, and Shian Zhang
Wavefront shaping enables the transformation of disordered speckles into ordered optical foci through active modulation, offering a promising approach for optical imaging and information delivery. However, practical implementation faces significant challenges, particularly due to the dynamic variation of speckles over time, which necessitates the development of fast wavefront shaping systems. This study presents a coded self-referencing wavefront shaping system capable of fast wavefront measurement and control. By encoding both signal and reference lights within a single beam to probe complex media, this method addresses key limitations of previous approaches, such as interference noise in interferometric holography, loss of controllable elements in coaxial interferometry, and the computational burden of non-holographic phase retrieval. Experimentally, we demonstrated optical focusing through complex media, including unfixed multimode fibers and stacked ground glass diffusers. The system achieved runtime of 21.90 and 76.26 ms for 256 and 1024 controllable elements with full-field modulation, respectively, with corresponding average mode time of 85.54 and 74.47 µs—pushing the system to its hardware limits. The system’s robustness against dynamic scattering was further demonstrated by focusing light through moving diffusers with the correlation time as short as 21 ms. These results emphasize the potential of this system for real-time applications in optical imaging, communication, and sensing, particularly in complex and dynamic scattering environments.
Advanced Imaging
  • Publication Date: Feb. 19, 2025
  • Vol. 2, Issue 1, 011002 (2025)
Privacy-preserving face recognition with a mask-encoded microlens array
Shukai Wu, Zheng Huang, Caihua Zhang, Conghe Wang, and Hongwei Chen
With advancements in artificial intelligence, face recognition technology has significantly improved in accuracy and reliability, yet concerns over privacy and data security persist. Currently, methods for addressing privacy issues focus on software and hardware levels, facing challenges in system power consumption, computational complexity, and recognition performance. We propose a novel privacy-preserving face recognition system that safeguards privacy at the optical level before signals reach the sensor. This approach employs a mask-encoded microlens array for optical convolution, effectively protecting privacy while enabling feature extraction for face recognition with a backend electronic neural network. Based on this passive optical convolution implementation under incoherent illumination, our system achieves superior spatial resolution, enhances light throughput, and improves recognition performance. An end-to-end training strategy with a dual cosine similarity-based loss function balances privacy protection and recognition performance. Our system demonstrates a recognition accuracy of 95.0% in simulation and 92.3% in physical experiments, validating its effectiveness and practical applicability.
Advanced Imaging
  • Publication Date: Jan. 24, 2025
  • Vol. 2, Issue 1, 011001 (2025)
Differential high-speed aperture-coding light field microscopy for dynamic sample observation with enhanced contrast
Junzheng Peng, Suyi Huang, Jianping Li, Xuejia He, Manhong Yao, Shiping Li, and Jingang Zhong
Light field microscopy can obtain the light field’s spatial distribution and propagation direction, offering new perspectives for biological research. However, microlens array-based light field microscopy sacrifices spatial resolution for angular resolution, while aperture-coding-based light field microscopy sacrifices temporal resolution for angular resolution. In this study, we propose a differential high-speed aperture-coding light field microscopy for dynamic sample observation. Our method employs a high-speed spatial light modulator (SLM) and a high-speed camera to accelerate the coding and image acquisition rate. Additionally, our method employs an undersampling strategy to further enhance the temporal resolution without compromising the depth of field (DOF) of results in light field imaging, and no iterative optimization is needed in the reconstruction process. By incorporating a differential aperture-coding mechanism, we effectively reduce the direct current (DC) background, enhancing the reconstructed images’ contrast. Experimental results demonstrate that our method can capture the dynamics of biological samples in volumes of 41 Hz, with an SLM refresh rate of 1340 Hz and a camera frame rate of 1340 frame/s, using an objective lens with a numerical aperture of 0.3 and a magnification of 10. Our approach paves the way for achieving high spatial resolution and high contrast volumetric imaging of dynamic samples.
Advanced Imaging
  • Publication Date: Dec. 06, 2024
  • Vol. 1, Issue 3, 031002 (2024)
Multi-polarization fusion network for ghost imaging through dynamic scattering media|Editors' Pick , Author Presentation
Xin Lu, Zhe Sun, Yifan Chen, Tong Tian, Qinghua Huang, and Xuelong Li
In this study, we propose a ghost imaging method capable of penetrating dynamic scattering media through a multi-polarization fusion mutual supervision network (MPFNet). The MPFNet effectively processes one-dimensional light intensity signals collected under both linear and circular polarization illumination. By employing a multi-branch fusion architecture, the network excels at extracting multi-scale features and capturing contextual information. Additionally, a multi-branch spatial-channel cross-attention module optimizes the fusion of multi-branch feature information between the encoder and the decoder. This synergistic fusion of reconstruction results from both polarization states yields reconstructed object images with significantly enhanced fidelity compared to ground truth. Moreover, leveraging the underlying physical model and utilizing the collected one-dimensional light intensity signal as the supervisory labels, our method obviates the need for pre-training, ensuring robust performance even in challenging, highly scattering environments. Extensive experiments conducted on free-space and underwater environments have demonstrated that the proposed method holds significant promise for advancing high-quality ghost imaging through dynamic scattering media.
Advanced Imaging
  • Publication Date: Dec. 09, 2024
  • Vol. 1, Issue 3, 031001 (2024)
Photonic timestamped confocal microscopy|On the Cover
Siyuan Yin, Shibao Wu, Zhanming Li, Haoran Lu, Zhiyao Wang, Zengquan Yan, and Xianmin Jin
Confocal microscopy, as an advanced imaging technique for increasing optical resolution and contrast, has diverse applications ranging from biomedical imaging to industrial detection. However, the focused energy on the samples would bleach fluorescent substances and damage illuminated tissues, which hinders the observation and presentation of natural processes in microscopic imaging. Here, we propose a photonic timestamped confocal microscopy (PT-Confocal) scheme to rebuild the image with limited photons per pixel. By reducing the optical flux to the single-photon level and timestamping these emission photons, we experimentally realize PT-Confocal with only the first 10 fluorescent photons. We achieve the high-quality reconstructed result by optimizing the limited photons with maximum-likelihood estimation, discrete wavelet transform, and a deep-learning algorithm. PT-Confocal treats signals as a stream of photons and utilizes timestamps carried by a small number of photons to reconstruct their spatial properties, demonstrating multi-channel and three-dimensional capacity in the majority of biological application scenarios. Our results open a new perspective in ultralow-flux confocal microscopy and pave the way for revealing inaccessible phenomena in delicate biological samples or dim life systems.
Advanced Imaging
  • Publication Date: Oct. 28, 2024
  • Vol. 1, Issue 2, 021005 (2024)