Research Article|13 Article(s)
Privacy-preserving face recognition with a mask-encoded microlens array
Shukai Wu, Zheng Huang, Caihua Zhang, Conghe Wang, and Hongwei Chen
With advancements in artificial intelligence, face recognition technology has significantly improved in accuracy and reliability, yet concerns over privacy and data security persist. Currently, methods for addressing privacy issues focus on software and hardware levels, facing challenges in system power consumption, computational complexity, and recognition performance. We propose a novel privacy-preserving face recognition system that safeguards privacy at the optical level before signals reach the sensor. This approach employs a mask-encoded microlens array for optical convolution, effectively protecting privacy while enabling feature extraction for face recognition with a backend electronic neural network. Based on this passive optical convolution implementation under incoherent illumination, our system achieves superior spatial resolution, enhances light throughput, and improves recognition performance. An end-to-end training strategy with a dual cosine similarity-based loss function balances privacy protection and recognition performance. Our system demonstrates a recognition accuracy of 95.0% in simulation and 92.3% in physical experiments, validating its effectiveness and practical applicability.
Advanced Imaging
  • Publication Date: Jan. 24, 2025
  • Vol. 2, Issue 1, 011001 (2025)
Differential high-speed aperture-coding light field microscopy for dynamic sample observation with enhanced contrast
Junzheng Peng, Suyi Huang, Jianping Li, Xuejia He, Manhong Yao, Shiping Li, and Jingang Zhong
Light field microscopy can obtain the light field’s spatial distribution and propagation direction, offering new perspectives for biological research. However, microlens array-based light field microscopy sacrifices spatial resolution for angular resolution, while aperture-coding-based light field microscopy sacrifices temporal resolution for angular resolution. In this study, we propose a differential high-speed aperture-coding light field microscopy for dynamic sample observation. Our method employs a high-speed spatial light modulator (SLM) and a high-speed camera to accelerate the coding and image acquisition rate. Additionally, our method employs an undersampling strategy to further enhance the temporal resolution without compromising the depth of field (DOF) of results in light field imaging, and no iterative optimization is needed in the reconstruction process. By incorporating a differential aperture-coding mechanism, we effectively reduce the direct current (DC) background, enhancing the reconstructed images’ contrast. Experimental results demonstrate that our method can capture the dynamics of biological samples in volumes of 41 Hz, with an SLM refresh rate of 1340 Hz and a camera frame rate of 1340 frame/s, using an objective lens with a numerical aperture of 0.3 and a magnification of 10. Our approach paves the way for achieving high spatial resolution and high contrast volumetric imaging of dynamic samples.
Advanced Imaging
  • Publication Date: Dec. 06, 2024
  • Vol. 1, Issue 3, 031002 (2024)
Multi-polarization fusion network for ghost imaging through dynamic scattering media|Editors' Pick
Xin Lu, Zhe Sun, Yifan Chen, Tong Tian, Qinghua Huang, and Xuelong Li
In this study, we propose a ghost imaging method capable of penetrating dynamic scattering media through a multi-polarization fusion mutual supervision network (MPFNet). The MPFNet effectively processes one-dimensional light intensity signals collected under both linear and circular polarization illumination. By employing a multi-branch fusion architecture, the network excels at extracting multi-scale features and capturing contextual information. Additionally, a multi-branch spatial-channel cross-attention module optimizes the fusion of multi-branch feature information between the encoder and the decoder. This synergistic fusion of reconstruction results from both polarization states yields reconstructed object images with significantly enhanced fidelity compared to ground truth. Moreover, leveraging the underlying physical model and utilizing the collected one-dimensional light intensity signal as the supervisory labels, our method obviates the need for pre-training, ensuring robust performance even in challenging, highly scattering environments. Extensive experiments conducted on free-space and underwater environments have demonstrated that the proposed method holds significant promise for advancing high-quality ghost imaging through dynamic scattering media.
Advanced Imaging
  • Publication Date: Dec. 09, 2024
  • Vol. 1, Issue 3, 031001 (2024)
Photonic timestamped confocal microscopy|On the Cover
Siyuan Yin, Shibao Wu, Zhanming Li, Haoran Lu, Zhiyao Wang, Zengquan Yan, and Xianmin Jin
Confocal microscopy, as an advanced imaging technique for increasing optical resolution and contrast, has diverse applications ranging from biomedical imaging to industrial detection. However, the focused energy on the samples would bleach fluorescent substances and damage illuminated tissues, which hinders the observation and presentation of natural processes in microscopic imaging. Here, we propose a photonic timestamped confocal microscopy (PT-Confocal) scheme to rebuild the image with limited photons per pixel. By reducing the optical flux to the single-photon level and timestamping these emission photons, we experimentally realize PT-Confocal with only the first 10 fluorescent photons. We achieve the high-quality reconstructed result by optimizing the limited photons with maximum-likelihood estimation, discrete wavelet transform, and a deep-learning algorithm. PT-Confocal treats signals as a stream of photons and utilizes timestamps carried by a small number of photons to reconstruct their spatial properties, demonstrating multi-channel and three-dimensional capacity in the majority of biological application scenarios. Our results open a new perspective in ultralow-flux confocal microscopy and pave the way for revealing inaccessible phenomena in delicate biological samples or dim life systems.
Advanced Imaging
  • Publication Date: Oct. 28, 2024
  • Vol. 1, Issue 2, 021005 (2024)
Real-time 3D imaging based on ROI fringe projection and a lightweight phase-estimation network
Yueyang Li, Junfei Shen, Zhoujie Wu, Yajun Wang, and Qican Zhang
Realizing real-time and highly accurate three-dimensional (3D) imaging of dynamic scenes presents a fundamental challenge across various fields, including online monitoring and augmented reality. Currently, traditional phase-shifting profilometry (PSP) and Fourier transform profilometry (FTP) methods struggle to balance accuracy and measurement efficiency simultaneously, while deep-learning-based 3D imaging approaches lack in terms of speed and flexibility. To address these challenges, we proposed a real-time method of 3D imaging based on region of interest (ROI) fringe projection and a lightweight phase-estimation network, in which an ROI fringe projection strategy was adopted to increase the fringe period on the tested surface. A phase-estimation network (PE-Net) assisted by phase estimation was presented to ensure both phase accuracy and inference speed, and a modified heterodyne phase unwrapping method (MHPU) was used to enable flexible phase unwrapping for the final 3D imaging outputs. The experimental results demonstrate that the proposed workflow achieves 3D imaging with a speed of 100 frame/s and a root mean square (RMS) error of less than 0.031 mm, providing a real-time solution with high accuracy, efficiency, and flexibility.
Advanced Imaging
  • Publication Date: Sep. 23, 2024
  • Vol. 1, Issue 2, 021004 (2024)
Ultra-robust imaging restoration of intrinsic deterioration in graded-index imaging systems enabled by classified-cascaded convolutional neural networks
Zaipeng Duan, Yang Yang, Ruiqi Zhou, Jie Ma, Jiong Xiao, Zihang Liu, Feifei Hao, Jinwei Zeng, and Jian Wang
Endoscopic imaging is crucial for minimally invasive observation of biological tissues. Notably, the integration between the graded-index (GRIN) waveguides and convolutional neural networks (CNNs) has shown promise in enhancing endoscopy quality thanks to their synergistic combination of hardware-based dispersion suppression and software-based imaging restoration. However, conventional CNNs are typically ineffective against diverse intrinsic distortions in real-life imaging systems, limiting their use in rectifying extrinsic distortions. This issue is particularly urgent in wide-spectrum GRIN endoscopes, where the random variation in their equivalent optical lengths leads to catastrophic imaging distortion. To address this problem, we propose a novel network architecture termed the classified-cascaded CNN (CC-CNN), which comprises a virtual-real discrimination network and a physical-aberration correction network, tailored to distinct physical sources under prior knowledge. The CC-CNN, by aligning its processing logic with physical reality, achieves high-fidelity intrinsic distortion correction for GRIN systems, even with limited training data. Our experiment demonstrates that complex distortions from multiple random-length GRIN systems can be effectively restored using a single CC-CNN. This research offers insights into next-generation GRIN-based endoscopic systems and highlights the untapped potential of CC-CNNs designed under the guidance of categorized physical models.
Advanced Imaging
  • Publication Date: Sep. 19, 2024
  • Vol. 1, Issue 2, 021003 (2024)
Block-modulating video compression: an ultralow complexity image compression encoder for resource-limited platforms
Siming Zheng, Yujia Xue, Waleed Tahir, Zhengjue Wang, Hao Zhang, Ziyi Meng, Gang Qu, Siwei Ma, and Xin Yuan
Considering the image (video) compression on resource-limited platforms, we propose an ultralow-cost image encoder, named block-modulating video compression (BMVC) with an extremely low-cost encoder to be implemented on mobile platforms with low consumption of power and computation resources. Accordingly, we also develop two types of BMVC decoders, implemented by deep neural networks. The first BMVC decoder is based on the plug-and-play algorithm, which is flexible with different compression ratios. The second decoder is a memory-efficient end-to-end convolutional neural network, which aims for real-time decoding. Extensive results on the high-definition images and videos demonstrate the superior performance of the proposed codec and the robustness against bit quantization.
Advanced Imaging
  • Publication Date: Aug. 07, 2024
  • Vol. 1, Issue 2, 021002 (2024)
High-throughput, nondestructive, and low-cost histological imaging with deep-learning-assisted UV microscopy
Jiajie Wu, Weixing Dai, Claudia T. K. Lo, Lauren W. K. Tsui, and Terence T. W. Wong
Pathological examination is essential for cancer diagnosis. Frozen sectioning has been the gold standard for intraoperative tissue assessment, which, however, is hampered by its laborious processing steps and often provides inadequate tissue slide quality. To address these limitations, we developed a deep-learning-assisted, ultraviolet light-emitting diode (UV-LED) microscope for label-free and slide-free tissue imaging. Using UV-based light-sheet (UV-LS) imaging mode as the learning target, UV-LED images with high contrast are generated by employing a weakly supervised network for contrast enhancement. With our approach, the image acquisition speed for providing contrast-enhanced UV-LED (CE-LED) images is 47 s/cm2, ∼25 times faster than that of the UV-LS system. The results show that this approach significantly enhances the image quality of UV-LED, revealing essential tissue structures in cancerous samples. The resulting CE-LED offers a low-cost, nondestructive, and high-throughput alternative histological imaging technique for intraoperative cancer detection.
Advanced Imaging
  • Publication Date: Aug. 07, 2024
  • Vol. 1, Issue 2, 021001 (2024)
Snapshot macroscopic Fourier ptychography: far-field synthetic aperture imaging via illumination multiplexing and camera array acquisition|On the Cover , Author Presentation
Sheng Li, Bowen Wang, Haitao Guan, Qian Chen, and Chao Zuo
Fourier ptychography (FP) is an advanced computational imaging technique that offers high resolution and a large field of view for microscopy. By illuminating the sample at varied angles in a microscope setup, FP performs phase retrieval and synthetic aperture construction without the need for interferometry. Extending its utility, FP’s principles can be adeptly applied to far-field scenarios, enabling super-resolution remote sensing through camera scanning. However, a critical prerequisite for successful FP reconstruction is the need for data redundancy in the Fourier domain, which necessitates dozens or hundreds of raw images to achieve a converged solution. Here, we introduce a macroscopic Fourier ptychographic imaging system with high temporal resolution, termed illumination-multiplexed snapshot synthetic aperture imaging (IMSS-SAI). In IMSS-SAI, we employ a 5×5 monochromatic camera array to acquire low-resolution object images under three-wavelength illuminations, facilitating the capture of a high spatial-bandwidth product ptychogram dataset in a snapshot. By employing a state-multiplexed ptychographic algorithm in IMSS-SAI, we effectively separate distinct coherent states from their incoherent summations, enhancing the Fourier spectrum overlap for ptychographic reconstruction. We validate the snapshot capability by imaging both dynamic events and static targets. The experimental results demonstrate that IMSS-SAI achieves a fourfold resolution enhancement in a single shot, whereas conventional macroscopic FP requires hundreds of consecutive image recordings. The proposed IMSS-SAI system enables resolution enhancement within the speed limit of a camera, facilitating real-time imaging of macroscopic targets with diffuse reflectance properties.
Advanced Imaging
  • Publication Date: Jun. 06, 2024
  • Vol. 1, Issue 1, 011005 (2024)
Label-free super-resolution stimulated Raman scattering imaging of biomedical specimens
Julien Guilbert, Awoke Negash, Simon Labouesse, Sylvain Gigan, Anne Sentenac, and Hilton B. de Aguiar
Far-field super-resolution microscopy has unraveled the molecular machinery of biological systems that tolerate fluorescence labeling. Conversely, stimulated Raman scattering (SRS) microscopy provides chemically selective high-speed imaging in a label-free manner by exploiting the intrinsic vibrational properties of specimens. Even though there were various proposals for enabling far-field super-resolution Raman microscopy, the demonstration of a technique compatible with imaging opaque biological specimens has been so far elusive. Here, we demonstrate a single-pixel-based scheme, combined with robust structured illumination, that enables super-resolution in SRS microscopy. The methodology is straightforward to implement and provides label-free super-resolution imaging of thick specimens, therefore paving the way for probing complex biological systems when exogenous labeling is challenging.
Advanced Imaging
  • Publication Date: Jun. 11, 2024
  • Vol. 1, Issue 1, 011004 (2024)