Journals >Chinese Journal of Lasers
Known for its non-invasive and non-destructive nature, optical microscopy can provide structural and functional insights into biological specimens, thus driving progress in fields such as biology, medicine, and related disciplines. Over the past four centuries, optical microscopy has witnessed significant developments. These have been particularly accelerated in the last century by technological advancements in lasers and computational methods. These advancements have led to revolutionary changes, making optical microscopy an essential tool in critical sectors such as healthcare, education, and food safety. With the increasing exploration in cellular biology and biomedicine, a growing need has arisen for optical microscopes with molecular or nanoscale spatial resolution, as exemplified by super-resolution optical microscopy (SRM). Of the various SRM techniques, stimulated emission depletion (STED) microscopy stands out because it achieves resolution enhancement by modulating the depletion power relative to the redshift in the excitation wavelength in the imaging setup. However, excessive power depletion poses challenges, including photobleaching of fluorophores and phototoxicity to biological specimens, which constrain the utility of STED in live-cell imaging scenarios. In recent years, researchers worldwide have collaborated to advance the field of STED microscopy with a particular focus on developing strategies to reduce depletion power. This effectively decreases the amount of power required for imaging while maintaining resolution accuracy. These studies are crucial for understanding the intricate details and underlying mechanisms in living organisms.
Progress In this review, we discuss the basic principles of STED microscopy and emphasize its crucial role in achieving super-resolution imaging of biological samples. Achieving super-resolution imaging using STED microscopy requires precise control over the spatial, temporal, and spectral aspects (Fig. 1). By applying the theoretical framework that governs the resolution calculations in STED microscopy, we outline methods for achieving low-power STED microscopy from four key perspectives: optimizing STED probes, using single-molecule localization techniques, employing advanced image processing methods, and utilizing time-resolved detection approaches.
We then provide a brief summary of the current nanoprobes designed for low-power STED imaging that encompass organic molecule dyes and organic and inorganic nanomaterials. Based on a comparative analysis of their performance parameters and imaging outcomes, we highlight the essential criteria for nanoprobes suitable for STED imaging, with a focus on attributes such as photobleaching resistance, low saturation intensity, and favorable biocompatibility. We also summarize and compare the imaging capabilities of STED microscopy and its derivative technologies. Noteworthy examples include MINFLUX, LocSTED, and MINSTED, which synergistically combine the strengths of STED and single-molecule localization microscopy (SMLM) to achieve substantial enhancements in imaging resolution (Fig. 2). In terms of image processing, we expound on the principles of differential image processing, explaining its effectiveness in modulating fluorescence signals across the spatial, temporal, and frequency domains to facilitate low-power STED imaging (Fig. 3). Moreover, by leveraging the insights into the relationship between the stimulated emission effect and fluorescence lifetimes, we advocate for the adoption of time-resolved detection modules to discern fluorescence photons with long lifetimes. Through techniques such as time-gated detection, phasor plot analysis, and ratiometric photon reassignment, we demonstrate the potential for enhanced resolution by selectively isolating photons with prolonged lifetimes (Fig. 4). Finally, we evaluate the prevailing challenges impeding the widespread adoption of low-power STED microscopy, emphasizing the need for future research endeavors that optimize image quality and enhance both imaging depth and the intelligence and automation of imaging systems. Our primary objective is to advance the application of STED microscopy, particularly in demanding domains such as thick-tissue imaging and in vivo investigations.
Conclusions and Prospects In the field of super-resolution imaging, STED microscopy is a pioneering far-field technique distinguished by its real-time capabilities, ultra-high resolution, and three-dimensional layer-slicing capabilities. These attributes make STED microscopy highly promising for bioimaging applications. To extend the utility of STED microscopy to in vivo imaging scenarios, a primary objective is to effectively reduce the depletion power, which is a major focus for future advancements in STED microscopy. With continuing advancements in scientific technology and the increasing demand for various applications, low-power STED microscopy enhancements are anticipated to progress further. For example, tailoring imaging parameters to diverse experimental conditions can be facilitated by integrating artificial intelligence and machine learning methodologies. This facilitates automatic parameter matching and the identification and tracking of target structures, thereby mitigating the complexity associated with experimental operations and enhancing both imaging efficiency and accuracy. In addition, the integration of STED microscopy with complementary advanced technologies holds promise for realizing expanded capabilities, including large-depth, multicolor, and three-dimensional imaging. These advancements are expected to provide researchers in the fields of biology and medicine with powerful tools for understanding complex biological processes.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107101 (2024)
The application of deep neural networks to image segmentation is one of the most prevalent topics in medical imaging. As an initial step in computer-aided detection processes, medical image segmentation aims to identify contours or regions of interest within images, thereby providing valuable assistance to clinicians in image interpretation, surgical planning, and clinical decision-making. Deep neural networks, which leverage their powerful ability to learn complex image features, have demonstrated outstanding performance in medical image segmentation. However, the use of deep neural networks for medical image segmentation has two significant limitations. First, different medical imaging modalities and specific segmentation tasks exhibit diverse image characteristics, leading to the low generalization capabilities of deep neural networks, which are often tailored to specific tasks. Second, increasingly complex network architectures with notable segmentation efficacy demand significant amounts of annotated image data, particularly those that require laborious manual annotation by medical experts.
With the rapid advancement of large-scale pretrained foundation models (LPFMs) in the field of artificial intelligence, an increasing number of tasks have achieved superior results through the fine-tuning of LPFMs. LPFMs are generic models trained on massive amounts of data and acquire foundational and versatile representational capabilities that can be transferred across different domains. Consequently, various downstream tasks can be easily fine-tuned using universal models. Considering the challenges in medical image segmentation, including low model generalization and difficulty in dataset acquisition, universal LPFMs are urgently needed in the field of medical image segmentation to facilitate breakthroughs in artificial intelligence applied to medical imaging.
Since its introduction as a foundational large model in the field of natural image segmentation, the segment anything model (SAM) has been applied across various domains with remarkable results. Although SAM has demonstrated powerful capabilities in natural image segmentation, its direct application to medical image segmentation tasks has yielded less-than-satisfactory outcomes. This can be attributed to two main factors. First, the training datasets contain shortcomings. SAM lacks sufficient representation of medical images in its training data, and medical images often exhibit blurry edges, which differ significantly from the clear edges present in natural images. Second, the characteristics of SAM prompts play a crucial role in segmentation performance. Only by judiciously selecting prompt strategies can the full potential of SAM be realized.
For these two reasons, significant efforts have been directed toward fine-tuning SAM, adapting SAM to three-dimensional (3D) medical datasets, expanding SAM functionalities, and optimizing prompting strategies. Comprehensive review articles have summarized these endeavors, such as the study by Zhang et al., which extensively outlined advancements in fine-tuning SAM, expanding its functionalities, optimizing prompting strategies, and distilling the challenges faced by SAM in the field of medical image segmentation. However, a systematic summary of methods for applying SAM to 3D medical datasets is lacking. Zhang et al. primarily elaborated on the fine-tuning of SAM, its application to 3D medical datasets, and related automatic prompting strategies. Nevertheless, as research on SAM deepens and its performance across various datasets improves, efforts in fine-tuning SAM, adapting it to 3D datasets, and optimizing prompting strategies have become more sophisticated. In addition, SAM has been extended to integrate semi-supervised learning methods and has been applied to novel directions such as interactive clinical healthcare. To summarize comprehensively the progress of SAM adaptation to medical image segmentation as well as to address existing challenges and provide directions for further research, a review that specifically focuses on the application of SAM to medical image segmentation is essential.
Progress This study extensively reviewed more than one hundred articles focusing on the utilization of SAM for medical image segmentation. Initially, this study furnished an exhaustive exposition of the SAM architecture and delineated its direct application to medical image datasets (Table 1). Then, an in-depth analysis of SAM's adaptation to medical image segmentation was conducted, emphasizing innovative refinements in fine-tuning techniques, SAM's integration into 3D medical datasets, and its amalgamation with semi-supervised learning methodologies (Fig. 3) alongside other emerging avenues. Experimental evaluations on two proprietary medical image datasets validated the enhanced generalization capabilities of the large models after extensive data fine-tuning (Table 2). In addition, the study confirmed the effectiveness of combining SAM with semi-supervised networks in generating high-quality pseudo-labels, thereby augmenting the segmentation performance (Table 3). Finally, the study delved into the current limitations, identified areas requiring improvement, elucidated the challenges encountered in SAM's adaptation to medical image segmentation, and proposed future directions, including the construction of large-scale datasets, enhancement of multi-modal and multi-scale information processing, integration of SAM with semi-supervised network structures, and expansion of SAM's application in clinical settings.
Conclusions and Prospects SAM is progressively being established as a potent asset in the field of medical image segmentation. In summary, although the integration of SAM into medical image segmentation holds great promise, it continues to face many challenges. Addressing these challenges requires a more comprehensive investigation and more refined approach, thus paving the way for effective implementation and further evolution of large-scale models in the domain of medical segmentation.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107102 (2024)
Structured illumination microscopy (SIM) is a pivotal technique in super-resolution microscopy as it offers an innovative approach to enhance the spatial resolution exceedingly beyond that achievable by conventional optical microscopes. SIM harnesses the principle of structured illumination, where finely patterned light interacts with the specimen, thereby generating moiré fringes containing high-frequency information that is otherwise unaccessible owing to the diffraction limit.
Achieving genuine super-resolution via SIM is involves intricate steps, including capturing numerous low-resolution images under an array of varied illumination patterns. Each of these images encapsulates a unique set of moiré patterns, which serve as the foundation for the subsequent computational reconstruction of a high-resolution image. Although effective, this methodology presents some challenges. Biological samples, owing to their inherent irregularities and varying tissue thicknesses, can result in considerable variability in the quality and consistency of the captured moiré patterns. This variability hinders the accurate reconstruction of high-resolution images. Additionally, systematic errors can further complicate the process, thus potentially introducing artifacts or resulting in the loss of crucial details in the final image.
Furthermore, sample damage due to prolonged light exposure must be considered when acquiring multiple images. Hence, the number of images required must be minimized without compromising the quality of the super-resolution reconstruction. Determining the optimal balance between the number of images and the quality of the final image is key in applying SIM to sensitive biological samples.
Image-processing algorithms are widely employed to mitigate the effect of excessive image pairs on imaging results. In addition to the classical algorithms, recently developed deep-learning algorithms offer promising solutions. Deep-learning algorithms can extract meaningful information from limited data and efficiently reconstruct images using neural networks. This approach enables high-quality super-resolution images to be acquired faster without necessitating numerous input images. Consequently, in SIM image reconstruction, satisfactory results can be achieved using fewer input images. Furthermore, deep-learning algorithms can effectively manage irregularities and variations in samples. By learning the structure and features of samples, these algorithms can better adapt to different types of samples, thus improving the robustness and accuracy of image reconstruction. This is particularly important when managing complex biological samples, which typically exhibit diversity and variability. Therefore, analyzing and summarizing the applications and effectiveness of deep learning in SIM systems is crucial.
Progress In deep learning, the widely recognized efficient neural network models include the convolutional neural network (CNN), U-Net, and generative adversarial network (GAN). The CNN, which is renowned for its capacity to automatically discern patterns and features within intricate datasets, is particularly suitable for the task mentioned above. By undergoing rigorous training on a substantial corpus of SIM images, the CNN learns to infer missing information that would otherwise require an array of supplementary images to capture. This predictive prowess enables the algorithm to amend the aberrations induced by SIM mode adjustments, thus significantly improving the quality of the reconstructed images. Because of the strategic deployment of skip connections within U-Net, which ingeniously amalgamates information from both the deeper and shallower layers, the network can effectively preserve abundant details and information throughout the upsampling phase. Furthermore, the integration of deconvolution processes not only amplifies the dimensions of the output image but is also pivotal in enhancing U-Net’s exceptional performance and widespread acceptance within the biomedical sector. In the context of SIM reconstruction, harnessing U-Net to extract supplementary insights from available images allows the algorithm to construct high-resolution images from a minimal subset of input images, thereby considerably diminishing the likelihood of specimen damage. By employing U-Net, one can reconstruct a super-resolved image similar to those afforded by classical algorithms using only three captured images. Furthermore, the implementation of GANs has significantly augmented the capabilities of deep-learning algorithms in SIM image processing. GANs comprise two dueling neural networks—a generator and a discriminator—that operate in tandem to fabricate highly realistic images. The generator synthesizes the images, whereas the discriminator assesses their veracity. Similar to U-Net, GANs can reconstruct super-resolved images from three original images. However, GANs can generate data through adversarial learning, and when coupled with other architectures, they can achieve even better results.
In summary, to enhance performance and generate high-resolution images from a minimal number of original images, various neural network models are synergistically combined. Finally, the application of deep learning in nonstriped and non-super-resolution SIM yields encouraging results, thereby further expanding the possibility of its applicability.
Conclusions and Prospects The integration of deep-learning algorithms into SIM image processing significantly advances the microscopy field. It not only addresses the technical challenges associated with achieving super-resolution but also provides new possibilities for investigating the nanoscale world with unprecedented clarity and detail. As deep-learning algorithms continue to advance, we expect more sophisticated algorithms to emerge and thus transcend the current boundaries of super-resolution microscopy.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107103 (2024)
In the dynamic field of biomedical photonics, simulating light transport in biological tissues has become a cornerstone for advancing medical diagnostics, therapeutic interventions, and understanding photobiological processes. This research area is crucial due to its potential to transform a wide range of biomedical applications. These include high-resolution medical imaging technologies, such as optical coherence tomography and fluorescence imaging, and innovative therapeutic approaches such as photodynamic therapy. These simulations provide detailed insights into the complex interactions between light and biological tissues, enhancing the precision of medical diagnostics, allowing for tailored light-based treatments for individual patients, and furthering our understanding of light-induced biological effects.
Monte Carlo (MC) simulation methods are at the forefront of this field, noted for their unparalleled flexibility and accuracy in modeling the stochastic nature of photon transport through media with diverse optical properties. The MC approach excels at replicating the complex phenomena of absorption, scattering, reflection, and refraction that characterize light’s interaction with heterogeneous biological tissues. Its ability to theoretically achieve any desired level of precision establishes it as the gold standard for simulating complex tissue optics scenarios, providing a crucial benchmark for validating results from other modeling techniques.
However, the practical use of MC simulations is significantly hindered by their high computational demands, which require extended periods to produce accurate results. This limitation not only affects the method’s efficiency but also presents a major barrier to its application in real-time or high-throughput settings. Consequently, there is a pressing need for innovative acceleration techniques that can reduce the computational load of MC simulations without sacrificing accuracy. Developing and implementing such strategies is essential to broaden the use and impact of photon transport simulations in biomedical research and clinical practice, facilitating quicker and more precise analyses that can advance medical science and improve patient care.
Progress In recent years, the field of MC simulations for photon transport has witnessed significant advancements aimed at overcoming the computational intensity that characterizes traditional MC methods. These innovations have led to substantially faster simulations, enhancing the practical applicability of MC techniques in biomedical photonics. Advances in this area include algorithmic improvements, the adoption of parallel computing strategies, and the development of specialized hardware accelerators.
Firstly, advancements in algorithms have led to the development of modified MC methods that maintain accuracy while significantly reducing computation times. Techniques, such as the baseline simulation method, adjust parameters, such as photon quantity and scattering characteristics, to accelerate the process. Perturbation MC methods introduce minor changes to existing simulations to evaluate the impact of alterations in optical properties without needing a complete re-simulation. Hybrid approaches merge traditional MC simulations with analytical calculations, such as the diffusion approximation, balancing speed with accuracy. Additionally, variance reduction techniques, such as importance sampling and path length trimming, have been crucial in minimizing statistical fluctuations, thereby enhancing the precision of the simulation outcomes.
Secondly, the integration of parallel computing techniques represents a significant advancement. The use of multicore CPUs and GPUs for parallel processing has transformed the field, allowing multiple simulations to run simultaneously. This development has not only drastically reduced computation times but also alleviated constraints related to the complexity of tissue models. Since the introduction of GPU-accelerated MC simulations in 2009, there has been a noticeable increase in research activity in this domain, reflecting a growing preference for parallel computing among researchers. The scalability of these technologies enables MC simulations to be executed on computer clusters, providing vast potential for addressing large-scale and complex simulation tasks.
Lastly, the design and implementation of specialized hardware for accelerating MC simulations have shown promising results, particularly in energy efficiency and performance within computation-constrained environments. Although the development pace of these dedicated hardware accelerators lags behind that of general-purpose processors, they represent a forward-thinking solution capable of supporting mobile monitoring and photonic control applications.
These advancements in MC simulation techniques not only signify substantial progress in the field but also underscore the collaborative efforts of the global scientific community. Institutions in China, the United States, France, and Germany have made notable contributions. As these technologies continue to advance, they promise to further improve the accuracy, efficiency, and practical applicability of photon transport simulations in biomedical research and clinical settings.
Conclusions and Prospects The advancements in acceleration techniques for MC simulations have effectively addressed the inherent limitations of classical MC methods, particularly their computational intensity, thus broadening their use in various areas of biomedical photonics. Accelerated algorithms, parallel computing strategies, and specialized hardware have each been crucial in improving the efficiency and feasibility of MC simulations for modeling light transport in biological tissues. These developments have not only enabled faster simulations but have also maintained, and in some instances improve, the accuracy and reliability of the results.
Looking forward, the ongoing evolution of computing technologies and algorithms promises significant further advancements in MC simulation acceleration. The integration of artificial intelligence and machine learning could revolutionize, for example, could offer novel approaches to optimize simulation parameters and predict outcomes, reducing computational demands. Additionally, the growing availability of high-performance computing resources and cloud platforms is set to democratize advanced MC simulations, making them more accessible to researchers and clinicians globally. As the field advances, the key challenge will be balancing computational efficiency with accuracy to ensure that accelerated MC simulations remain a robust tool for examining the intricate interactions between light and biological tissues. The future of MC simulation in biomedical photonics is promising, poised to substantially enhance medical diagnostics, therapy planning, and our understanding of photobiological processes.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107104 (2024)
Due to its economic advantages, convenience of use, and wide applicability, fluorescence fluctuation-based super-resolution microscopy has rapidly advanced in recent years and has garnered increased attention and application. Compared with other super-resolution imaging techniques, fluorescence fluctuation-based super-resolution microscopy offers lower system costs and is particularly suitable for imaging live cells, demonstrating exceptional performance in observing subcellular structures and monitoring dynamic processes. Specifically, variations in the fluorescence fluctuation characteristics significantly affect the quality of the super-resolution reconstructed images. Therefore, a systematic investigation of image quality under various fluorescence fluctuation conditions is crucial for identifying the most suitable super-resolution imaging approach. These fluorescence fluctuation conditions include parameters such as the number of image-acquisition frames, signal-to-noise ratio, bright-to-dark state probability, and bright-to-dark fluorescence intensity ratio, which directly affect image clarity, the signal-to-noise ratio, and accuracy. Thoroughly examining these conditions, we can effectively select and optimize the super-resolution imaging method that meet specific research requirements and experimental conditions.
We developed a fluorescence fluctuation-based super-resolution comprehensive imaging reconstruction platform using MATLAB. This platform integrates four super-resolution methods, namely, SOFI, MSSR, MUSICAL, and SPARCOM, and can simulate fluorescence fluctuation signals under different conditions while simultaneously applying multiple super-resolution methods to generate datasets. The platform also supports the import and reconstruction of experimental data and presents the reconstruction results clearly and intuitively on the platform interface, thus allowing users to conveniently compare the imaging results of different approaches. A comprehensive image-quality assessment is then conducted on these simulated datasets. This study used four sets of data under different fluorescence fluctuation conditions and quantitatively analyzed the quality of the reconstructed images generated by the four super-resolution algorithms using five evaluation parameters: the resolution-scaled Pearson coefficient (RSP), resolution-scaled error (RSE), relative error of strength (K), signal-to-noise ratio (SNR), and resolution (R). These five parameters were used to determine the image reconstruction consistency, reconstruction error, image reconstruction uniformity, SNR of the reconstructed images, and improvements in the reconstructed image resolution. In addition, to assess the quality of images reconstructed by the super-resolution algorithms more comprehensively and objectively, this study assigned specific weights to these five evaluation parameters and defined a comprehensive evaluation factor (CEF). The weights were determined based on the relative importance of each parameter in the super-resolution imaging technology to ensure the contribution of each parameter was accurately reflected. To facilitate a better comparison of the performances of the four super-resolution algorithms, this study integrated a multilayer perceptron model with a CEF and datasets generated under various fluorescence fluctuation conditions. The model can determine the super-resolution image reconstruction method that best performs under various fluorescence fluctuation conditions by learning and analyzing the performance of different algorithms and outputting an optimal algorithm selection. In short, this model considers different fluorescence fluctuation conditions as inputs and uses a comprehensive evaluation factor of the reconstructed results from various super-resolution algorithms as outputs.
Under the fluorescence fluctuation super-resolution comprehensive imaging reconstruction platform, fluorescence signals under varying fluorescence fluctuation conditions were generated. Super-resolution algorithms were applied to reconstruct the datasets and calculate their CEF values; some simulation results are presented in Table 1. The SPARCOM method demonstrates the best performance in terms of resolution and denoising capability, achieving a spatial resolution of up to 44 nm. However, this method relies heavily on the sparsity of image sequences for super-resolution reconstruction and struggles to reconstruct images accurately when the bright-state probability of the fluorescence fluctuation signal is too high or the bright-dark ratio is too low. The MUSICAL method, which has lower resolution capabilities, offers superior denoising performance but exhibits poor image reconstruction consistency, uniformity, and a longer reconstruction time. The MSSR method has moderate resolution capabilities but exhibits superior image reconstruction consistency and uniformity and can be combined with other super-resolution algorithms to obtain higher-quality super-resolution images. Although the SOFI method has lower resolution and denoising capabilities, it exhibits good image reconstruction consistency and uniformity and exhibits a higher image reconstruction rate. A multi-layer perceptron model was constructed with fluorescence fluctuation characteristics as inputs and the CEF values of different algorithms as outputs. An analysis of the generated and evaluated datasets showed that the constructed model achieves an accuracy of 92.3%, indicating reliable classification and recognition capabilities and enabling intelligent selection of the most suitable super-resolution image reconstruction method under varying fluorescence fluctuation signal conditions.
We developed a comprehensive super-resolution image reconstruction platform using MATLAB, which implements signal generation and super-resolution image reconstruction functions under various fluorescence fluctuation conditions. The performances of multiple super-resolution algorithms across different fluorescence fluctuation scenarios were systematically evaluated. Leveraging of the dataset generated by the software platform enabled us to introduce a multi-layer perceptron model for intelligent algorithm selection. This in turn allowed for accurate classification and identification of the optimal super-resolution technique. This approach enhances research efficiency and assists researchers in selecting the most suitable fluorescence fluctuation method for various subcellular super-resolution imaging studies. The approach can further advance the application of fluorescence fluctuation-based super-resolution imaging techniques for efficient investigation of the ultrafine structures of various biological subcellular organelles.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107105 (2024)
Fluorescence molecular tomography (FMT), which can observe the three-dimensional distribution of fluorescent probes in small animals via reconstruction algorithms, has become a promising imaging technology for preclinical studies. The strong scattering property of biological tissues and limited boundary measurements with noise have resulted in the FMT reconstruction problem being severely ill-posed. To solve the problem of FMT reconstruction, some studies have been conducted from different aspects, e.g., the improvement of forward modeling and many regularization-based algorithms. Owing to the ill-posed nature and sensitivity to noise of the inverse problem, it is a challenge to develop a robust algorithm that can accurately reconstruct the location and morphology of the fluorescence source. Traditional reconstruction algorithms use the
In this study, we applied the Huber iterative hard threshold (HIHT) algorithm to fluorescence molecular tomography. The HIHT algorithm modifies the
Numerous numerical simulations and in vivo mouse experiments are conducted to evaluate the performance of the HIHT algorithm. The reconstruction performance of the HIHT algorithm is illustrated by the contrast-to-noise ratio (CNR), Dice coefficient, location error (LE), normalized mean square error (NMSE), and time. Quantitative and qualitative analyses show that the HIHT algorithm achieves the best reconstruction results in terms of the localization accuracy, spatial resolution of the fluorescent source, and morphological recovery, compared with the FISTA, Homotopy, and IVTCG algorithms (Figs. 1, 4). To further verify the robustness of the HIHT algorithm, we perform four sets of experiments with different Poisson and Gaussian noise intensities (Fig. 2 and Fig. 3). As the noise intensity increases, the NMSE of the HIHT algorithm is always the smallest, indicating that it has the highest reconstruction accuracy. At the same noise intensity, the HIHT algorithm has the smallest LE, indicating that it reconstructs the target closest to the position of the real source. When the noise intensity increases, the Dice coefficient of the HIHT algorithm is higher than those of the other three algorithms, which indicates that the HIHT algorithm has a better morphological reconstruction ability. The CNR fluctuation of the HIHT algorithm is smaller than the CNR variations of the other three algorithms in the 10%?25% noise range. The results show that when the noise level is lower than 25%, the HIHT algorithm still obtains satisfactory reconstruction results, compared with those of the other three algorithms. To further evaluate the reconstruction performance of the HIHT algorithm in practical applications, we also perform in vivo mouse experiments. The experimental results show that the HIHT algorithm has the smallest position error as well as the highest Dice coefficient, and the fluorescent bead reconstructed by the HIHT algorithm is the closest to the real fluorescent bead in terms of morphology, which further demonstrates the feasibility and robustness of the HIHT algorithm (Fig. 5). The experimental results show that the HIHT algorithm not only achieves accurate fluorescence target reconstruction, but also improves the robustness to noise.
This study investigates the problem of insufficient algorithm robustness in FMT, and the HIHT algorithm reduces the impact of noise on the reconstruction performance by using the Huber loss function as the residual term. With the same noise intensity, compared with the other three algorithms, the HIHT algorithm obtains the smallest LE and NMSE values as well as the largest CNR and Dice coefficient values, indicating that the HIHT algorithm has the best reconstruction performance. As the noise intensity increases, the reconstruction performance of the HIHT algorithm outperforms the other three algorithms, and the performance is more superior in the Poisson noise test, which indicates that the HIHT algorithm has the best reconstruction accuracy and robustness. The experimental results are consistent with the theoretical description in Section 2. These results indicate that the HIHT algorithm is insensitive to noise and has good robustness. In summary, when the measurement data sets are disturbed by noise, unlike the algorithms based on the
- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107106 (2024)
Diabetic retinopathy (DR) is one of the most common complications of diabetes and one of the main causes of irreversible vision impairment or permanent blindness among the working-age population. Early detection has been shown to slow the disease's progression and prevent vision loss. Fundus photography is a widely used modality for DR-related lesion identification and large-scale screening owing to its non-invasive and cost-effective characteristics. Ophthalmologists typically observe fundus lesions, including microaneurysms (MAs), hemorrhages (HEs), hard exudates (EXs), and soft exudates (SEs), in images to perform manual DR diagnosis and grading for all suspected patients. However, expert identification of these lesions is cumbersome, time consuming, and easily affected by individual expertise and clinical experience. With the increasing prevalence of DR, automated segmentation methods are urgently required to identify multiclass fundus lesions. Recently, deep-learning technology, which is represented by convolutional neural networks (CNNs) and Transformers, has progressed significantly in the domain of medical-image analysis and has become the mainstream technology for DR-related lesion segmentation. The most commonly used methods are semantic segmentation-oriented CNNs, Transformers, or their combinations. These deep-learning methods exhibit promising results in terms of both accuracy and efficiency. Nevertheless, CNN-based methods are inferior in terms of global-character contextual information owing to their intrinsically limited receptive field, whereas Transformer-based approaches exhibit low local inductive biases and subpar perception of multiscale feature dependencies. Whereas models combining CNNs with transformers exhibit clear advantages, they require the extraction of deep semantic characteristics and direct feature concatenation from the same feature level without fully considering the importance of concrete boundary information for small-lesion segmentation, thus resulting in inadequate feature interaction between adjacent layers and conflicts among different feature scales. Moreover, these methods only focus on a certain type of DR lesion and seldom delineate multitype lesions simultaneously, thereby hampering their practical clinical application.
In this study, we developed a novel progressive multifeature fusion network based on an encoder-decoder U-shaped structure, which we named PMFF-Net, to achieve accurate multiclass DR-related fundus lesion segmentation. The overall framework of the proposed PMFF-Net is shown in Fig. 1. It primarily comprises an encoder module embedding a hybrid Transformer (HT) module, a gradual characteristic fusion (GCF) module, a selective edge aggregation (SEA) module, a dynamic attention (DA) module, and a decoder module. For the encoder module, we sequentially cascaded four HT blocks to form four stages to excavate multiscale long-range features and local spatial information. For a fundus image
We used two publicly available DR datasets, i.e., IDRiD and DDR, to verify the proposed PMFF-Net. The comparison results (see Tables 1 and 2) show that our PMFF-Net performs better than the current state-of-the-art DR lesion-segmentation models on the two datasets, with mDice and mIoU values of approximately 45.11% and 33.39%, respectively, for predicting EX, HE, MA, and SE simultaneously on the IDRiD dataset; and mDice and mIoU values of 36.64% and 35.04%, respectively, on the DDR dataset. Specifically, compared with H2Former, our model achieves higher mDice and mIoU values by 3.94 percentage points and 3.28 percentage points, respectively, on the IDRiD dataset, and 4.55 percentage points and 4.69 percentage points higher values, respectively, compared with those of PMCNet. On the DDR dataset, our model achieves the best segmentation results, outperforming H2Former by 5.17 percentage points and 6.15 percentage points in terms of mDice and mIoU, respectively, and surpassing PMCNet by 6.36 percentage points and 7.43 percentage points, respectively. Meanwhile, our model can provide real-time DR-lesion analysis, with analysis times of approximately 34.74 and 38.48 ms per image on the IDRiD and DDR datasets, respectively. The visualized comparison results shown in Figs. 6 and 7 indicate that the results predicted by our model are more similar to the ground truth compared with those of other advanced methods. The cross-validation results across datasets presented in Tables 3 and 4 show that, compared with other advanced segmentation methods, our model offers better generalizability. The perfect segmentation performance of the developed PMFF-Net may be attributed to the ability of our HT module in capturing global context information and local spatial details, the GCF module gradually aggregating different levels of multiscale features through high-level semantic information guidance, the DA module eliminating irrelevant noise and enhancing DR-lesion discriminative feature identification, and the SEA block establishing a constraint between the DR-lesion region and boundary. Additionally, the effectiveness of the components of the proposed PMFF-Net was justified, including the HT, GCF, DA, and SEA modules, on the IDRiD dataset.
In this study, we developed a novel PMFF-Net for the simultaneous segmentation of four types of DR lesions in retinal fundus images. In the PMFF-Net, we constructed an HT module by elegantly integrating a CNN, multiscale channel attention, and Transformer to model the long-range global dependency of lesions and their local spatial features. The GCF module was designed to merge features from adjacent encoder layers progressively under the guidance of high-level semantic cues. We utilized a DA module to suppress irrelevant noisy interference and refine the fusion multiscale features from the GCF module dynamically. Furthermore, we incorporated an SEA module to emphasize lesion boundary contours and recalibrate lesion locations. Extensive experimental results on the IDRiD and DDR datasets show that our PMFF-Net perform better than other competitive segmentation methods. By performing cross-validation across datasets, the excellent generalizability of our model can be similarly demonstrated. Finally, we demonstrated the effectiveness and necessity of the proposed model via a comprehensive ablation analysis. The developed method can serve as a general segmentation framework and has been applied to segment other types of biomedical images.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107107 (2024)
The fundus is the only part of the human body where arteries, veins, and capillaries can be directly observed. Information on the vascular structure of the retina plays an important role in the diagnosis of fundus diseases and exhibits a close relationship with systemic vascular diseases such as diabetes, hypertension, and cardiovascular and cerebrovascular diseases. The accurate segmentation of blood vessels in retinal images can aid in analyzing the geometric parameters of retinal blood vessels and consequently evaluating systemic diseases. Deep learning algorithms have strong adaptability and generalization and have been widely used in fundus retinal blood vessel segmentation in recent years. Digital image processing technology based on deep learning can extract blood vessels from fundus images more quickly; however, the contrast of fundus images is mostly low at the boundary of blood vessels and microvasculature, and the extraction error of blood vessels is large. In particular, the microvasculature, which is similar in color to the background and has a smaller diameter, renders it more difficult to extract less vascular areas from the background. To solve this problem, this study improves the classical medical-image semantic segmentation U-Net. To effectively extract the spatial context information of color fundus images, a multiscale feature mixing and fusion module is designed to alleviate the limitations of local feature extraction by the convolution kernel. Moreover, to solve the problem of low contrast of the microvessels in color fundus images, a microvessel feature extraction auxiliary network is designed to facilitate the network in learning more detailed microvessel information and improve the performance of the network's blood vessel segmentation.
A microvascular segmentation model of a parallel network based on U-Net (MPU-Net) was designed based on microvascular detail information loss and limitations of the convolution kernels. The U-Net network model was improved. First, the U-Net network was paralleled with an auxiliary network for microvascular feature extraction (Mic-Net). Microvascular labels on the gold standard images of fundus blood vessels were obtained via morphological processing, and they were used in the auxiliary network of microvascular feature extraction to learn microvascular feature information. Second, the main network was introduced in a multiscale feature shuffling and fusion module (MSF). Through learning, more receptive field characteristic information can be used to relieve the convolution kernels under space limitations. In contrast, a channel-mixing mechanism was used to increase the interaction between channels to better integrate the characteristics of different receptive field sizes and microvascular characteristics. MPU-Net comprised two parallel U-Net branches: the main and microvascular feature extraction auxiliary networks. The network that used the whole blood vessel label to calculate the loss function is the main network, whereas the Mic-Net used the microvessel label to calculate the loss function. Each network branch had one lesser layer of upper sampling on the U-Net architecture to reduce the loss of detail. A multiscale feature shuffle fusion module was introduced into the main network to alleviate the limitation of obtaining local information by convolution and to fuse microvessel feature information into the main network more effectively. In this study, a multi-scale feature-mixing fusion module MSF was designed. The module had two input features. The first was the encoder output feature, which contained more spatial details and exhibited a better expression ability on thick blood vessels. The other was the decoder feature or the microvascular feature output by the decoder in Mic-Net, which contained more high-level semantic information.
We use three publicly available datasets—DRIVE, CHASE_DB1, and STARE—to validate the proposed MPU-Net. The comparison results (see Table 1, Table 2 and Table 3) show that the MPU-Net proposed in this study performs well in terms of accuracy. As presented in Table 1, for the DRIVE test set, the accuracy, sensitivity, specificity, and AUC of the proposed MPU-Net are 0.9710, 0.8243, 0.9853, and 0.9889, respectively. Compared with existing segmentation method (TDCAU-Net), MPU-Net obtains the highest accuracy, sensitivity, specificity, and AUC, which are improved by 0.0154, 0.0056, 0.0097, and 0.0094, respectively. Further, compared with DG-Net, which exhibits a better overall segmentation performance, MPU-Net increases the values by 0.0106, 0.0629, 0.0016, and 0.0043, respectively. These results indicate that the MPU-Net proposed in this study performs well on the DRIVE dataset and is beneficial for improving the vascular segmentation accuracy of the DRIVE dataset from the perspective of microvascular feature extraction and multi-scale feature wash-and-wash fusion. As presented in Table 2, for the CHASE_DB1 test set, the accuracy, sensitivity, specificity, and AUC of the proposed MPU-Net are 0.9764, 0.8593, 0.9844, and 0.9913, respectively. Compared with the existing segmentation method (TDCAU-Net), MPU-Net obtains the highest accuracy, sensitivity, and AUC, which are increased by 0.0026, 0.0350, 0.0035, respectively. Further, compared with ACCA-MLA-D-U-Net, which exhibits a better sensitivity performance, it increases the values by 0.0091, 0.0191, and 0.0039, respectively. These results show that MPU-Net has a better segmentation performance on the CHASE_DB1 datasets, although the performance of MPU-Net is slightly lower than that reported by Mao et al. on specificity, but 0.0352 and 0.0020 higher than that in sensitivity and AUC, respectively. As shown in Table 3, for the STARE test set, the proposed MPU-Net values are 0.9768, 0.7844, 0.9907, and 0.9905 for accuracy, sensitivity, specificity, and AUC, respectively. Compared with the existing segmentation method (LUVS-Net), MPU-Net obtains the highest accuracy, specificity, and AUC, which are increased by 0.0015, 0.0046, and 0.1718, respectively. Further, compared with CS2-Net, which had the best sensitivity performance, it increases the values by 0.0098, 0.0094, and 0.0030, respectively. These results show that the proposed MPU-Net is better than the existing mainstream methods in terms of accuracy, specificity, and AUC, but the performance in the sensitivity index is not sufficiently good. In addition, there is a certain gap compared with CS2-Net, but the other indicators are better than those of CS2-Net. This indicates that on the STARE dataset, the model algorithm is significantly affected by the imbalance of vascular pixels and background pixel samples, and will improve the specificity by sacrificing the sensitivity. However, from the perspective of the overall evaluation indices of the model, namely accuracy and AUC, the MPU-Net model exhibits better performance. Further, from the perspective of the overall segmentation performance, MPU-Net is superior to the existing mainstream methods on the STARE dataset. This proves that it is helpful for the overall segmentation performance on the STARE dataset from the perspective of microvascular feature extraction and multi-scale feature shuffling and fusion. From the analysis of the three datasets, MPU-Net is confirmed to be better than the existing mainstream methods in terms of the accuracy and AUC indicators, indicating that the proposed method is beneficial for improving the overall segmentation performance of the model and has a certain generalization ability. For both the DRIVE and CHASE_DB1 datasets, the sensitivity index is superior to existing mainstream methods, indicating that the MPU-Net model can further improve the segmentation sensitivity of blood vessels. Thus, this study effectively improves the vascular segmentation performance of color fundus images from the perspectives of microvascular feature extraction and multiscale feature mixing and fusion.
In this study, from the perspective of retinal vascular segmentation, microvascular lesions are found to have an important reference value for systemic vascular diseases diagnose. However, there are still certain difficulties in microvascular segmentation tasks. Therefore, in the vascular segmentation task, the shortcomings of deep convolutional neural network for microvascular segmentation are studied, and a parallel network microvascular segmentation model based on U-Net is proposed for vascular segmentation tasks. To alleviate the limitations of feature extraction of convolutional neural networks, a multiscale feature-shuffling fusion module is used to exploit the feature information extracted by the convolutional neural network, and the continuity of vascular segmentation is effectively improved by increasing the interaction between channels and combining spatial multiscale information. To alleviate the loss of detailed information during feature extraction caused by the pooling operation in the U-Net encoder, a microvascular feature extraction auxiliary network was proposed to further extract microvascular feature information. The test results for the DRIVE, CHASE_DB1, and STARE validation sets demonstrate that the proposed network can effectively improve the vascular segmentation performance compared with existing networks with better performance. In the future, further research should be conducted based on the auxiliary network of microvascular feature extraction to extract more refined and comprehensive microvascular features.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107108 (2024)
Medical image registration is a spatial transformation process that aligns and matches the specific spatial structures contained in two medical images. It has been applied in disease detection, surgical diagnosis and treatment, and other fields. Traditional medical image registration methods are slow and computationally expensive. In recent years, researchers have made significant breakthroughs in medical image registration research using deep learning methods. Deep learning methods have increased the registration speed by hundreds of times, with a registration accuracy comparable to those of traditional methods. However, most patients have complex pathological conditions and lesions grow quickly, resulting in significant differences in the images collected at different stages. Existing deep learning-based registration methods have low registration accuracy and poor generalization performance when used for medical images of large deformations. Therefore, a multi-scale constraint network (MC-Net) for large-deformation 3D medical image registration based on multi-scale constraints is proposed.
We propose a multi-scale constraint network (MC-Net) for large-deformation 3D medical image registration based on multi-scale constraints. Three multi-kernel (MK) modules are designed as parallel multi-channel and multi-convolution kernels for the encoder to accelerate the training speed. A convolutional block attention module (CBAM) is added to skip connections and enhance the ability to extract complex semantic information and fine-grained feature information from large-deformation images. In order to improve the registration accuracy, MC-Net combines multi-scale constrained loss functions to implement a layer-by-layer optimization strategy from low resolution to high resolution.
In an experiment, three publicly available 3D datasets (OASIS, LPBA40, and Abdomen CT-CT, with two modalities) were used for registration research. The effectiveness of MC-Net was demonstrated through original experiments, traditional comparison methods, deep learning comparison methods, ablation experiments, and multi-core fusion experiments. Based on the registration results shown in Figs. 5 and 6, MC-Net performed well in the registration of the OASIS and LPBA40 brain datasets, as well as for the Abdomen CT-CT abdominal dataset. In the brain image comparison experiment, the LPBA40 brain dataset was compared with a traditional registration method (ANTs) and three deep learning registration methods (VoxelMorph, CycleMorph, and TransMorph) in the same experimental environment. It was found that MC-Net outperformed the other methods in terms of detail registration in brain regions and overall brain contour deformation. The abdominal image comparison experiment compared two traditional methods (ANTs and Elastix) and two deep learning methods (VoxelMorph and TransMorph). It was found that MC-Net had some shortcomings in organ generation and contour deformation, but had better registration performance than the other methods in terms of blank area size and individual organ deformation. The ablation experiment was conducted using the LPBA40 dataset. It demonstrated the different roles of the MK and CBAM modules in processing medical images in MC-Net, which helped to improve the registration accuracy. In addition, this article also discusses the computational complexity of MC-Net. For large target images such as medical images, this article discusses how a multi-kernel (MK) fusion module can be designed to effectively reduce the computational complexity.
In response to the low accuracy and poor generalization performance of current large-deformation image registration methods, this paper proposes a medical image registration network (MC-Net) based on multi-scale constraints, with LPBA40, OASIS, and Abdomen CT-CT medical image datasets used as research objects. Information loss can be avoided by designing CBAM modules in skip connections to enhance the ability to extract differential information from large-deformation images. In addition, considering the slow registration speed caused by the large number of parameters when processing large-deformation images, the MK module was designed with a parallel path large kernel convolution structure to improve the registration speed without affecting registration accuracy. When combined with the multi-scale constraint loss function proposed in this article, it iteratively optimizes the deformation fields at three scales from low resolution to high resolution to improve the registration accuracy. The experimental results show that compared with other methods, this method has improved registration accuracy, speed, and computational complexity. The good registration performances in three datasets with MRI and CT modalities demonstrate the generalization ability of our method. Subsequent research will focus on designing an adaptive adjustment module for multi-scale constrained loss function hyperparameters, in order to solve the problem of the time-consuming hyperparameter tuning needed for loss functions in experiments and improve the experimental efficiency. In summary, MC-Net has practical value in the registration of large-deformation images.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107109 (2024)
Brain tumors pose a significant threat to human health, and fully automatic magnetic resonance imaging (MRI) segmentation of brain tumors and their subregions is fundamental to their computer-aided clinical diagnosis. During brain MRI segmentation using deep learning networks, tumors occupy a small volume of medical images, have blurred boundaries, and may appear in any shape and location in the brain, presenting significant challenges to brain tumor segmentation tasks. In this study, the morphological and anatomical characteristics of brain tumors are integrated, and a UNet with a multimodal recombination module and scale cross attention (MR-SC-UNet) is proposed. In the MR-SC-UNet, a multitask segmentation framework is employed, and a multimodal feature recombination module is designed for segmenting different subregions, such as the whole tumor (WT), tumor core (TC), and enhancing tumor (ET). In addition, the learned weights are used to effectively integrate information from different modalities, thereby obtaining more targeted lesion features. This approach aligns with the idea that different MRI modalities highlight different subregions of brain tumor lesions.
To address the feature differences required for segmenting the different subregions of brain tumors, a segmentation framework was proposed in this study, which takes the segmentation task of three lesion regions as independent sub-tasks. In this framework, complementary and shared information among various modalities is fully considered, and a multimodal feature recombination module was designed to automatically learn the attention weights of each modality. The recombined features derived by integrating these learned attention weights with the traditionally extracted features are then input into the segmentation network. In the segmentation network, the module automatically learns the attention weights of each modality and recombines these weights with traditionally extracted features. By treating the segmentation tasks of the three lesion regions as independent subtasks, accurate segmentation of the gliomas is achieved, thereby addressing the problem of differing multimodal information requirements for different regions. To address the inability of a 3DUNet to fully extract global features and fuse multiscale information, a U-shaped network based on scale cross attention (SC-U-Net) was proposed. Specifically, a scale cross attention (SC) module was designed and incorporated into the deep skip connections of a 3DUNet. By leveraging the global modeling capability of the transformer model, SC extracts the global features of the image and fully integrates multiscale information.
Figure 7 shows the results of the ablation experiments with different configurations of the SC module. When the SC module is added to the 3rd to 5th skip connections, the network achieves the best integration of deep multiscale features, thereby enhancing the feature extraction capability of the model. The average Dice coefficient of the three regions reaches 87.98%, and the mean 95% Hausdorff distance is 5.82 mm, thereby achieving optimal performance. Table 1 lists the ablation experimental results. The best results are obtained when the proposed MR and SC modules are used together, with the Dice coefficients for the three subregions increased by 1.34, 2.33, and 7.08 percentage points. Table 2 presents the comparison results of the six state-of-the-art methods, indicating superior performance in most metrics. Figures 8 and 9 show the segmentation visualization results, revealing that the improved model can more accurately identify the tumor tissue, resulting in smoother segmentation boundaries. Additionally, by integrating multiscale features, the model gains a larger receptive field, reducing the unreasonable segmentation caused by a single-scale and limited receptive field. Therefore, the segmentation results are closer to the annotated images with minimal false-positive regions.
In this study, a deep learning network framework, MR-SC-UNet, is proposed and applied to glioma segmentation tasks. The test results on the BraTS2019 dataset show that the proposed method achieves average Dice scores of 91.13%, 87.46%, and 87.98% for the WT, TC, and ET regions, respectively, demonstrating its feasibility and effectiveness. In clinical applications, accurate tumor segmentation can significantly improve the capabilities of radiologists and neurosurgeons for disease assessment and provide a scientific basis for precise treatment planning and risk assessment of patients.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107110 (2024)
Surface-enhanced Raman spectroscopy (SERS) is an optical sensing technology based on local surface plasmon resonance, which greatly enhances the Raman signal of molecules adsorbed or very close to the surface of rough nano-metals, even achieving single-molecule detection. The traditional SERS substrate is created by depositing precious metal nanoparticles (Ag, Au, Cu, etc.) on rigid substrates such as slides and silicon wafers. The preparation process of such substrates is relatively mature, offering good stability and sensitivity, and is widely used in molecular recognition, quantitative analysis, and other fields. However, complicated experimental pretreatment steps, high cost, high operational requirements, and fixed detection platform shapes limit the application range of SERS technology, making it unsuitable for sampling and detection of objects with complex shapes and irregular surfaces. To improve the flexibility and portability of detection, reduce costs and operational requirements, and broaden the application range of SERS technology, researchers have focused on developing new SERS substrates, with flexible SERS substrates attracting significant attention. Flexible substrates possess good flexibility and plasticity, and can be cut to any desired shape and size to accommodate various complex shapes and irregular surfaces, offering great advantages in non-destructive and in-situ detection. This review introduces the research progress in SERS technology based on flexible substrates in recent years. Firstly, different methods of constructing flexible SERS substrates using various flexible materials, including cellulose flexible substrates, polymer flexible substrates and other flexible materials, are discussed, highlighting the advantages and challenges of each. Additionally, the latest applications of flexible SERS substrates in biomedicine, food safety, and environmental monitoring are summarized.
Progress First, based on the introduction of common materials for constructing flexible SERS substrates, including cellulose, polymer flexible films, and materials from natural organisms, this review outlines different methods for constructing SERS substrates under these materials and discusses their respective advantages and challenges. The natural 3D hotspot structure of cellulose makes it a reliable material for the green synthesis of nanoparticles and the manufacture of flexible SERS substrates. Cellulose paper-based SERS substrates have been widely studied due to their renewability and low cost (Fig. 2), with some unique preparation methods being highlighted (Fig. 3). Polymer films are extensively used in the SERS field due to their flexibility, transparency, and biocompatibility (Fig. 5). Additionally, various biological materials are increasingly attracting researchers’ attention due to their inherent properties or natural structures (Fig. 6). This study also reviews and summarizes recent applications of flexible SERS substrates in biomedicine, food safety, and environmental monitoring methods. Furthermore, the optimization strategies and challenges in various application scenarios based on flexible SERS substrates are summarized and anticipated.
Conclusions and Prospects Although flexible SERS substrates have been widely studied, challenges remain in their practical application. Material selection is critical, as uniformity and reproducibility of the SERS spectrum can be affected by varying pore sizes, chemical compositions, uneven distribution of reducing agents, and different aggregation states of plasma nanoparticles at different locations. The preparation process may involve the use of volatile organic solvents that are not environmentally friendly and can release harmful substances, leading to negative environmental effects. Differences between various organisms can impact experimental reproducibility. Adjusting the composition, morphology, and structure of nanoparticles, and introducing surface modification or pretreatment steps can improve substrate performance. Currently, most high-performance SERS substrates rely on precious metal nanostructures, and their high cost hinders mass production. Therefore, it is essential to explore more environmentally friendly, green, and high-performance SERS substrates, including the preparation of renewable substrates and the development of substrates with self-cleaning capabilities. In practical applications, the complex and diverse composition of objects presents a challenge for the multiplexed detection capability of flexible SERS substrates. Thus, combining SERS with other technologies such as molecular imprinted polymers (MIPs), immune recognition, microfluidic technology, and machine learning can help construct a SERS sensing platform suitable for multi-target detection. Furthermore, with the miniaturization of Raman spectrometers, SERS technology is expected to reduce dependence on large Raman spectroscopy instruments. Combined with flexible SERS substrates, SERS technology can potentially offer a new and rapid optical detection method for many special scenarios, including field exploration, emergency incident handling, criminal investigation, entry-exit border inspection, and clinical bedside detection.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107401 (2024)
Multispectral images, as captured using aerospace optical instruments, feature high spectral resolutions and abundant information. Hence, they are used extensively in the military, meteorology, mapping, and other fields. However, their large memory size poses significant transmission and storage challenges to remote-sensing satellites and end users. Image compression can solve this issue. The classical image encoding employs transform coding techniques to decompose the original image into coefficients concentrated in energy, which are then quantized to achieve efficient compression; however, this results in significant blocking artifacts. Currently, the performance of image compression based on the classical encoding is inferior to that of image compression based on deep-learning networks. In particular, end-to-end deep-learning models demonstrate excellent performance in terms of image compression. Nevertheless, most learning-based compression frameworks are designed for visible light and focus primarily on spatial redundancy, which results in suboptimal compression performance for multispectral images. Therefore, this study proposes a learning-based multispectral-image-compression network to address these challenges.
The network adopts a variational autoencoder architecture that incorporates rate-distortion optimization and a hyper-prior entropy model. Specifically, owing to the varying sensitivity of the human eye to information at different frequencies, the network initially employs a pooling convolutional network to decompose the input image into high- and low-frequency components. Subsequently, these components are input to separate feature-extraction networks for high and low frequencies. Feature extraction networks were constructed using the SSFE (spacial and spectral feature extraction), attention, and activation-function modules. Dense connections between layers are utilized to extract multiscale and contextual information on latent features across different frequencies. The extracted potential features are quantified and compressed into a bitstream using an arithmetic encoder. Simultaneously, the potential features are input to the prediction network of the hyper-prior entropy model to extract edge information and generate a probability-distribution model to facilitate decoding. The structure of the decoding end is symmetrical to that of the encoding end, and the feature components are restored to their original frequency components using the opposite operation. Finally, the dual-attention module integrates the high- and low-frequency components to generate reconstructed images, thus completing the compression process.
To verify the compression performance of the proposed compression method on multispectral images, we selected 8 and 12 band multispectral images for experiments, and the experimental datasets are both open-source datasets. The proposed method was compared with two classical encoded-image compression algorithms (JPEG2000 and 3D-SPHIT), a video-compression coding method (H.266/VCC), and two learning-based image-compression algorithms (Joint and DBSSFE-Net) using three evaluation indices: PSNR (peak signal-to-noise ration), MS-SSIM (multi-scale structral similarity index measurement), and MSA (mean spectral angle). The experimental results show that the proposed FDDBFE-Net yields higher PSNR values compared with various classical algorithms, with average improvements of 0.89 dB, 1.14 dB, and 1.87 dB compared with the DBSSFE, Joint, and VCC algorithms, respectively. Performance evaluation based on the MS-SSIM index shows that the proposed compression model is the most similar to the original image in terms of structural similarity, with improvements of 1.56 dB, 0.96 dB, and 2.95 dB compared with the DBSSFE, Joint, and VCC algorithms, respectively. Furthermore, the spectral-reconstruction quality shows that the proposed method provides the minimum spectral angle. This indicates that the reconstructed image has the smallest spectral loss and is the most similar to the original image in terms of quality. The proposed method exhibits lower network spectral losses by 13.1%, 9.5%, and 20.2% compared with the DBSSFE, Joint, and VCC algorithms, respectively. When compared with the results of the 12-band images, the disadvantages of the classical methods are particularly evident. Compared with DBSSFE-Net, the proposed algorithm yields a higher PSNR by 2.5 dB, a higher MS-SSIM by 2.2 dB, and a lower MSA by 30.6%. Compared with the Joint algorithm, it yields a higher PSNR by 0.9 dB, a higher MS-SSIM by 0.4 dB, and a lower MSA by 5.29%. Compared with the VCC algorithm, it yields a higher PSNR by 3.4 dB, a higher MS-SSIM by 3.9 dB, and a lower MSA by 34.9%. Additionally, the proposed algorithm demonstrates the optimal encoding and decoding time on a graphics processing unit (GPU), whereas its decoding time on a central processing unit (CPU) is longer, which is attributable to the frequency decomposition and synthesis modules added. In general, the proposed algorithm performs better than the other algorithms investigated in terms of compression performance.
In this study, a multispectral-image-compression network based on a variational autoencoder was proposed. The network has an end-to-end symmetrical structure and embeds various key technologies. Considering the spatial multiscale and spectral nonstationarity of multispectral images, a double-branch frequency decomposition feature extraction method was proposed, which can effectively extract the spatial and interspectral features of the images, enhance the attention to different channels, and improve the robustness of the model. Experimental results show that the proposed model achieves excellent performance on multispectral image datasets, which surpasses those of the conventional JPEG2000, 3D-SPHIT, and H.266/VCC compression methods. Furthermore, it performs better than the DBSSFE-Net and Joint algorithms, which are based on a variational autoencoder structure.
.- Publication Date: Oct. 29, 2024
- Vol. 51, Issue 21, 2109001 (2024)
During the fabrication of flip chips, challenges to production yield and longevity arise owing to preparation defects, including delamination, missing solder bumps, and cracks. These defects typically manifest at dimensions ranging from the submillimeter to micron scale and are characterized by pronounced randomness and broad distribution. Consequently, comprehensive defect detection across these multiscale dimensions facilitates the early identification and removal of flawed chips, thereby enhancing both the production yield and long-term operational stability. Whereas existing nondestructive testing methodologies such as ultrasonics, laser ultrasonics, X-ray computed tomography, pulsed phase thermography, and conventional photoacoustics partially satisfy the requirements of flip-chip detection, they exhibit certain challenges. These challenges include sample contamination, detection-speed limitations imposed by the average sampling procedures, potential risks associated with ionizing radiation, and susceptibility to environmental effects. Hence, this study introduces an intelligent defect-detection methodology based on non-interferometric noncontact photoacoustic microscopy (NINC-PAM). This approach is designed to achieve accurate and extensive detections of preparation defects across varying dimensions within flip chips. By offering a feasible technical solution for inline nondestructive defect detection during the fabrication process of flip chips, this methodology is promising for substantially improving both the production yield and operational lifespan of flip chips.
First, we established an NINC-PAM system based on elasto-optical theory and autonomously developed an optical?mechanical joint scanning imaging mode for the wide-field-of-view (FOV) imaging of flip-chip samples. Second, by leveraging the NINC-PAM system, we introduced a multiscale defect-detection algorithm named Chip-YOLO to identify preparation defects of varying sizes within flip chips. This algorithm enhances the original YOLOv8 architecture by sequentially incorporating a small-object detection (SOD) layer, large separable kernel attention (LSKA) module, and reparameterized generalized feature pyramid network (RepGFPN) to optimize the detection accuracy for defects of different sizes in flip chips. Third, the flip-chip samples were prepared in the following sequence: spin coating, dehydration, photolithography, development, evaporation, and lift-off. Delamination defects of various sizes were introduced into the samples via ultrasonic cleaning. Fourth, by employing the optical?mechanical joint scanning mode of the NINC-PAM system, the samples were subjected to multiple wide-FOV scans, thus resulting in a dataset containing 29706 defects of different sizes. Fifth, by utilizing a method based on absolute-scale target definition, the established dataset was statistically analyzed and quantified to classify defects of different sizes, thereby validating the rationality of the dataset.
The proposed wide-FOV multiscale intelligent defect-detection method, which is based on NINC-PAM, achieves the precise identification and localization of delamination defects across varying sizes within flip-chip samples on an extensive scale. Initially, by leveraging the dataset formulated from the NINC-PAM imaging results, the Chip-YOLO algorithm demonstrates a multiscale average accuracy (AP) of 60.1% under 12.4 MB of parameters and 39.8 GFLOPs of computation amount. This performance surpasses those of other classical detection algorithms, including YOLOv3, YOLOX, YOLOv7, and Faster R-CNN, in terms of both accuracy metrics and computational efficiency (Table 3). Subsequent ablation experimental findings reveal that the incorporation of the SOD layer, LSKA module, and RepGFPN into the foundational YOLOv8 architecture increases the multiscale AP of Chip-YOLO for delamination defects. Remarkably, without significantly increasing the model parameters or computational demand, Chip-YOLO achieves 3.3 percentage points enhancement in the multiscale AP relative to YOLOv8 (Table 4). More importantly, the experiment of wide-FOV multiscale intelligent detection based on the NINC-PAM system accomplished the intelligent detection of defects of various sizes in flip-chip samples exceeding 1 mm×1 mm in just 23 s. The detection accuracy of the proposed method demonstrates its more accurate defect-detection ability compared with other classical intelligent detection methods, thereby substantiating its superior performance in wide-FOV multiscale defect detection (Fig. 7). Moreover, the performance statistics show that Chip-YOLO offers more accurate and faster defect detections compared with other one- and two-stage algorithms (Table 5). In the current wide-FOV multiscale intelligent detection experiments, the time consumption of a single mechanical scan is on the order of hundreds of milliseconds, whereas that of a single optical scan is approximately 2 s, thus indicating that the experimental time is primarily governed by the galvanometer scanning process. Using a laser with megahertz repetition rates can increase the scanning frequency of the galvanometer for real-time imaging. Meanwhile, although Chip-YOLO offers better multiscale defect detection, it introduces additional parameters and computational overheads, thus prolonging the detection time. In the future, efficient CNN (convolutional neural networks) building blocks can be introduced to reduce the network’s parameter count and computational load for improving the detection speed. Furthermore, wide-FOV intelligent detection images are currently obtained by manually stitching multiple optical scan images after performing an experiment. Developing the appropriate intelligent stitching methods will allow real-time image stitching to be achieved during scanning and detection.
This study proposes a wide-FOV, multiscale intelligent defect-detection methodology based on the NINC-PAM system that is customized for the precise identification and intelligent localization of preparation defects of varying sizes in flip chips. This approach integrates the NINC-PAM system and operates in an optical–mechanical joint scanning mode with the proposed Chip-YOLO multiscale defect-detection algorithm to achieve accurate detections of delamination defects of different sizes within a wide FOV. On the established dataset of preparation defects in flip chips, Chip-YOLO achieves a multi-scale AP of 60.1%, surpassing the multiscale AP of other classical algorithms. Further ablation experimental results confirm that Chip-YOLO exhibits a higher multiscale AP compared with the original YOLOv8. More importantly, intelligent defect-detection experimental results prove that the proposed method can achieve accurate wide-FOV multiscale detection without increasing system latency. Similarly, performance statistics confirm that Chip-YOLO achieves more accurate and faster detections compared with other one- and two-stage algorithms. The proposed wide-FOV multiscale intelligent defect-detection method based on the NINC-PAM system demonstrates significant potential for online defect detection in flip chips.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2109002 (2024)
Inertial confinement fusion (ICF) is the primary approach for achieving controlled nuclear fusion. To enhance the efficiency of laser fusion energy and attain higher yields, Tabak proposed a fast-ignition scheme for ICF. A critical challenge in fast-ignition fusion schemes is precisely guiding multiple high-energy picosecond laser beams onto the target. Because the bottom circle of the target cone measures only tens of micrometers, an error of a few micrometers or preheating the cryogenic target will result in a failed ignition experiment. Consequently, the precision of beam?target coupling directly determines the result of the ICF experiment. This study presents a novel design for a target-alignment system for PW laser beams. The laser beams and target image are separated by a beam splitter; therefore, this design inhibits the direct laser irradiation of the cryogenic target. However, the alignment system enables high-precision beam–target coupling without direct laser irradiation, which benefits fast-ignition-scheme experiments.
Based on the laser-beam layout and the requirement of beam?target coupling in a fast-ignition scheme, the structure of the target-alignment system of picosecond PW laser beams was designed in this study; a schematic diagram of the system is shown in Figure 1. Based on the actual situation of beam?target coupling, the target-aiming system was constructed on an offline experimental platform; subsequently, the resolution test target was used to verify the imaging resolution of the system. A beam?target coupling experiment was conducted, and the beam–target coupling accuracy of this system was measured through multiple experiments. The Zemax software was used to optimize the imaging simulation of the target, and the resolution experiment was simulated and analyzed. Subsequently, the factor contributing to the low resolution was determined, and the imaging mirror set was designed and optimized to further improve the imaging quality. Finally, a program was designed to identify the positions of the target and focal spot in real time, thus improving the accuracy and efficiency of the beam?target coupling.
Results and discussions The resolution-verification experiment performed on this system shows that the initial plain plate splitter introduces severe aberration [Figs. 4(a) and (b)], which was solved after performing optimization using a cube plate splitter [Figs. 4(c) and (d)]. Further use of aspheric mirror improves the resolution, and the final resolution of the target in the experiment is 90.5 lp/mm (11.04 μm) in a 5 mm×5 mm field-of-view [Figs. 4(e) and (f)]. Figure 5 shows the beam?target coupling process using this system. The results of multiple experiments show that the error in the x-direction is slightly larger than that in the y-direction, which may be caused by the system asymmetry. Finally, the average position error of the beam?target coupling system is 4.32 μm (Table 1), which is lower than that of the existing beam?target coupling scheme and satisfies the high-precision beam?target coupling requirements. Zemax was used to verify the experimental results of target imaging, and the simulation analysis shows that the lens can be further optimized. Therefore, the lens was optimized, and the results are shown in Figure 7. The MTF (modulation transfer function) curve and simulated imaging results indicate that the resolution is greater than 5 μm in a 10 mm×10 mm field-of-view. Finally, a real-time recognition program was designed to identify the positions of the focal spot and target (Fig. 2), thus reducing the error of subjective judgment and improving the efficiency of beam?target coupling. Currently, this system can be further optimized. The mirror set optimized using Zemax has neither been processed nor adjusted, and further measurements of the target-imaging effect will be performed in subsequent studies.
This paper introduces an innovative target-alignment system for picosecond PW laser beams. To satisfy the requirements of cryogenic target coupling in an ICF fast-ignition experiment, beam?target coupling was realized via non-laser direct irradiation, which offers high levels of resolution, precision, and efficiency. An image-recognition program based on beam?target coupling was developed to avoid errors caused by manual judgment and to improve the efficiency of beam?target coupling. An offline verification experiment was performed, where the target resolution is 90.5 lp/mm (11.04 μm) in a 5 mm×5 mm field-of-view. Multiple experiments show that the average position error of beam?target coupling is 4.32 μm, which is lower than that of the existing remote-observation beam?target coupling scheme, which features tens of microns of error. Optical optimization was performed via simulation. After the optimization, the imaging resolution of the target is expected to be greater than 5 μm in a 5 mm×5 mm field-of-view. This system provides a high-precision solution for the picosecond laser-beam coupling of a cryogenic target and presents wide application prospects in future large-scale picosecond laser-beam experiments.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2101001 (2024)
Utilizing actively Q-switched fiber lasers to generate high-energy narrow-linewidth nanosecond laser pulses is a mainstream technology. Various types of active Q-switches have been developed based on different principles. Among them, fiber-pigtailed acousto-optic modulators (AOMs) and electro-optic modulators (EOMs) have garnered wide attention owing to their nanosecond-level switching time and high extinction ratios spanning tens of decibels. However, the output of Q-switched fiber lasers based on fast switches, such as AOMs and EOMs, typically exhibits a typical multipeak structure. Researchers have proposed various methods to address this issue; however, they present certain limitations. Therefore, obtaining high-power narrow-linewidth lasers with smooth pulses from actively Q-switched fiber lasers is crucial. In this study, a novel Q-switch is designed based on the linear electro-optic effect of a β-BaB2O4 (BBO) crystal with a high damage threshold. Moreover, to eliminate the multipeak phenomenon caused by the short rise time of the Q-switch, a fiber coupler is used to form a coupling ring (CR), which enables a gradual release of the initial signal energy and successfully realizes a smooth pulse output.
To construct an intensity modulator suitable for high-power lasers, a pair of single-mode fiber collimators (COLs) is used for spatial optical-path coupling. A high-damage-threshold BBO uniaxial crystal is placed between orthogonal polarization devices. The intensity of the transmitted laser can be modulated by applying a periodic half-wave voltage to the BBO crystal. Additionally, to obtain a smooth Q-switched pulse output, a smooth and long-rise waveform is required. The longer the circulation time within the fiber CR, the longer the rise time. However, this implies that the precision of the input pulse segmentation decreases, thereby reducing the smoothness of the rise-time waveform. Therefore, a fiber CR comprising optical coupler is inserted, with the ring length set to 0.5 m.
First, the performance of the constructed intensity modulator is tested (Fig. 2). The measured rise time is approximately 8 ns, with an insertion loss and extinction ratio of approximately 2 dB and 20 dB, respectively, thus satisfying the requirements of a high-gain double-cladding fiber laser for Q-switching. As the pump power increases [Fig. 3(a)], the repetition frequency of the pulse sequence gradually transitions from 1/4 to 1/3 and 1/2 of the modulation frequency, before eventually exhibiting a regular pulse sequence. This occurs because, after the laser pulse depletes inverted ions during cavity circulation, the gain generated in the doped fiber during the subsequent Q-switch closed period is insufficient to establish a pulse within the limited open time. By gradually increasing the pump power, the gain within a single cycle increases accordingly. When increased to a certain value, the accumulated gain within a single cycle is sufficient to support pulse establishment during the subsequent Q-switch open time, thus resulting in a pulse-sequence repetition frequency that matches the modulation frequency. The pulse-waveform details before and after the insertion of the fiber CR are shown in Fig. 3(b). After the Q-switch is activated, the waveform exhibits the typical multipeak characteristics before the insertion of the CR. After inserting the fiber CR, the multipeak structure on the pulse completely disappears and the temporal waveform becomes extremely smooth. The principle by which the fiber CR smooths the temporal waveform of the pulse is explained as follows: Considering the evolution process of a rectangular pulse passing through the fiber CR (Fig. 4), the analysis results summarized based on Equations (1) and (2) indicate that the original rectangular pulse, after transmission through the CR, evolves into a series of equally spaced delayed subpulses with different amplitudes, thus effectively altering the Q-switch rise time. Consequently, the temporal multipeak structure is completely eliminated, thus resulting in a smooth laser pulse. Additionally, the average power increases almost linearly with the pump power without any saturation effect [Fig. 5(a)]. As the damage threshold of the fiber devices and the pump power cannot be further increased to compress the pulse width, the duty cycle should be reduced to increase the gain. The width of the output pulse can be further compressed by reducing the duty cycle to increase the gain recovery time. When the duty cycle is reduced from 50% to 3%, the pulse width is compressed to 35 ns, and the pulse is extremely smooth with no multipeak structures. The laser output linewidth narrows to 0.08 nm with a signal-to-noise ratio of 60 dB [Fig. 5(b)].
In this study, a highly doped large-mode-area double-cladding fiber is used as the gain medium. Simultaneously, a novel intensity modulator based on the linear electro-optic effect of a high-damage-threshold BBO crystal is designed as a Q-switch. Furthermore, to eliminate multipeaks in the output pulse caused by the rapid opening of the Q-switch, a fiber CR is introduced to allow the initial signal energy to be released gradually, thereby resulting in smooth laser pulses. The principles of pulse temporal evolution are analyzed comprehensively. The laser yields a smooth pulse output with an average power exceeding 1 W, a linewidth as narrow as 0.08 nm, and a pulse width of 35 ns.
.- Publication Date: Nov. 05, 2024
- Vol. 51, Issue 21, 2101002 (2024)
The development of laser-cooling technology has advanced cold-atomic-frequency standards, thus enhancing the precision of defining a second to 10-16. Additionally, the second definition with a frequency uncertainty of 10-19, which offers even greater accuracy, is anticipated to be established in the optical-frequency standard. The cold-atomic-frequency standard offers a higher frequency accuracy and a lower frequency drift rate compared with the classical atomic-frequency standard, thereby providing higher long-term stability. Among the various physical factors affecting the accuracy of atomic-frequency standards, the distributed-cavity phase shift is one of the most important sources of uncertainty and a key factor to be considered in the miniaturization of cold-atomic-frequency standards. Currently, the analysis and calculation of the distributed-cavity phase shift primarily involves the Fourier decomposition of three-dimensional phase distributions in column coordinates. This method accurately solves the phase distribution of microwave cavities with simple structures and the corresponding frequency shifts. However, for phase distributions in rectangular cavities, annular cavities, and cylindrical cavities with multiple openings in the sidewalls, which are relatively complex, this method requires remodeling and is complicated. Rapid advancements in computer technology have enabled rapid three-dimensional finite-element simulations that accurately simulate the phase distribution in microwave cavities. This facilitates the optimization of microwave-cavity designs, thus reducing the effect of the distributed-cavity phase shift.
This study examines the phase distribution of an intracavity-cooling falling-detection atomic clock based on three-dimensional finite-element simulations. Initially, each atom is assigned a position based on a Gaussian distribution and specified with an initial velocity based on the Maxwell–Boltzmann distribution. The direction of the initial velocity is assumed to be completely random, and atomic collisions are disregarded. The sensitivity function is employed to calculate the change in the transition probability caused by each atom experiencing a phase variation, thus allowing for the determination of the total transition probability change and the distributed-cavity phase shift.
The simulation results (Fig. 3) show the phase distribution in a microwave cavity with and without sidewall opening. Apart from the slight phase-gradient change in the Y-direction caused by the structural asymmetry, the overall phase change is minimal. This suggests that opening the sidewall of the microwave cavity to introduce a laser field is feasible. Figure 4 provides a detailed analysis of the phase distribution inside the cavity. It reveals abrupt phase changes at approximately 20 mm in the radial direction. Additionally, the phase undulation within the cutoff waveguide is substantial, which results in significant distributed-cavity phase shifts when the atoms experience these phase changes. Figure 5 shows the calculated distributed-cavity phase shifts under different temperatures and Ramsey linewidths. When the Ramsey linewidths are constant, the motion range of the atoms within the microwave cavity increases with the atomic temperature. Consequently, the phase changes intensify, thus resulting in greater frequency shifts and uncertainties. However, under wider Ramsey linewidths, the distributed-cavity phase shifts are primarily caused by the position change of the atomic cluster due to gravity. The effect of the atomic-cluster temperature becomes more significant when the Ramsey linewidth is narrower. Under the same atomic temperature, the frequency uncertainty of the distributed-cavity phase shift decreases as the linewidth increases because of the inversely proportional relationship between time and the Ramsey linewidth for free evolution. Considering the structure and short-term stability of the microwave cavity, we observe that when the Ramsey linewidth is 10 Hz, the atomic clouds experience better phase shifts and power symmetry, and the uncertainty of the distributed-cavity phase shift is less than 2×10-16, thus indicating greater potential for long-term stability.
This study introduces a Monte Carlo method for traversing-atom analysis to calculate the distributed-cavity phase shift of microwave atomic clocks. This method is based on a three-dimensional finite-element simulation and is applied to cavity-cooled falling miniaturized atomic clocks. The results indicate that introducing laser light into the sidewall aperture minimally affects the intracavity phase distribution. Additionally, the intracavity-cooling atomic clocks do not experience significant phase variations near the aperture or inside the cutoff waveguide during operation, which is advantageous for evaluating the distributed-cavity phase shift. The frequency uncertainty degree is less than 2×10-16, which indicates that the long-term stability of the atomic clock is not limited by the distributed-cavity phase shift. This design for the miniaturization of atomic clocks is highly promising and is expected to offer more competitive time–frequency benchmarks with lower frequency uncertainties and better long-term stability compared with other miniaturized atomic clocks. The proposed calculation method can be extended to the distributed-cavity phase-shift calculation of rectangular cavities, loop-gap cavities, and other microwave cavities with complex structures, thus facilitating the microwave-cavity design of miniaturized atomic clocks.
.- Publication Date: Nov. 05, 2024
- Vol. 51, Issue 21, 2104001 (2024)
The “brain program”, which focuses strategically on neuroscience research, has been launched worldwide. The construction of neural connection maps is the basis of neuroscience research. Meanwhile, the analysis of neural circuits can provide a basis for investigations into the mechanisms of perception, memory, and social behavior, as well as the diagnosis and treatment of related diseases. Additionally, relevant investigations can inspire the development of next-generation artificial-intelligence algorithms and promote the development of the intelligent information industry.
Progress The development of optical technology and functional proteins in recent years has enabled scientists to use fluorescence imaging technology based on functional indicator proteins to observe neural activity, as well as use optogenetics based on opsins to regulate neural activity. All-optical physiology combines the observation and regulation of neural activity based on optical technology, which offers the advantages of low invasibility, high spatial resolution, and high throughput compared with conventional electrophysiological techniques, and has become an ideal method for the analysis of neural functional circuits in vivo.
In this paper, the technical route of all-optical physiological technology is first introduced (Fig. 1). In the second section, the principles and characteristics of the functional indicators of fluorescent and photosensitive proteins are introduced (Fig. 2). In the third section, the basic optical-path structure and technical realization method of all-optical physiological systems are reviewed (Fig. 3), and typical results based on single- and two-photon imaging are shown (Fig. 4). In the fourth section, the performance-evaluation indexes of all-optical physiological systems in the time and space dimensions are analyzed (Fig. 5), and recent improvements for enhancing the system performance are presented (Fig. 6).
Conclusions and Prospects The main research tools used in neuroscience have shifted from electrical to optical devices. All-optical physiological systems have been improved gradually. Integrating advanced technologies in the field of imaging and optogenetics enables the development of a mesoscopic multiphoton all-optical physiological system that can realize high-speed and accurate observations and regulations of hundreds of neurons in the entire brain region or several brain regions in three-dimensional space, analyze neural functional circuits, and construct a brain functional connection map.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107301 (2024)
Fluorescence lifetime imaging (FLIM) technology utilizes photoluminescence lifetime instead of intensity as a detection signal to effectively avoid autofluorescence interference from tissues, thereby providing enhanced imaging accuracy and comprehensive information regarding biochemical and cellular environments. The development of high-performance contrast agents is crucial for advancing FLIM technology. Currently, organic dye molecules are highly favored in FLIM owing to their tunable optical properties, good biocompatibility, and low synthesis costs. However, organic molecules tend to aggregate in biological environments, which renders it difficult to maintain a high fluorescence quantum yield and a long fluorescence lifetime, thus limiting their practical application. Therefore, high-performance dye molecules with satisfactory anti-quenching properties are urgently required to advance FLIM imaging in biomedical research. However, the rational molecular design of optimized fluorescence performance remains challenging owing to insufficient understanding regarding the excited-state dynamics within organic dye molecules. Excited-state dynamics is a critical aspect that correlates the dye molecule structure and macroscopic performance; thus, it determines the photophysical properties of the dye molecules. A comprehensive understanding and elucidation of the excited-state dynamics of dye molecules is crucial for guiding the design of high-performance fluorescent agents for FLIM applications.
Molecular steric hindrance was increased to optimize fluorescence performance. Nonetheless, comprehensive investigations into conformation-related excited-state dynamics allow one to elucidate the fundamental factors affecting fluorescence performance. Two boron-dipyrromethene (BODIPY) dyes with different steric hindrances were synthesized in this study. The core of classical BODIPY was decorated at the meso position with p-methylbenzene and trimethylbenzene to prepare P-BDP and MP-BDP, respectively. The optical properties of P-BDP and MP-BDP, including their spectra, quantum yield, and fluorescence lifetime, were analyzed using steady-state and time-resolved fluorescence spectroscopy. Quantum chemical calculations were performed to reveal the effect of molecular conformation on their optical properties. Furthermore, femtosecond transient absorption spectroscopy was performed to elucidate the effect of molecular conformation on the excited-state dynamics of the dyes, which fundamentally determines the optical properties of the molecules. Selected organic molecules with superior fluorescence performance were encapsulated in an amphiphilic copolymer (Pluronic F-127) to construct nanoparticles (NPs). Additionally, their potential for two-photon fluorescence imaging at the biological level was investigated.
The classic BODIPY features a boron-nitrogen heterocyclic core with a pyrrole ring attached to each side. This coplanar configuration of its tricyclic structure offers a significant π-conjugation effect, which imparts a high molar extinction coefficient. The introduction of the boron bridge not only restricts the free rotation and isomerization of the groups but also enhances the rigidity of the molecular structure, thus endowing BODIPY dyes with excellent fluorescence quantum yield and photostability. BODIPY derivatives, i.e., P-BDP and MP-BDP, with different steric hindrances were synthesized. The molecular structures were confirmed using 1H NMR and 13C NMR spectroscopies. Spectroscopic experiments and quantum chemical calculations indicate that steric hindrance minimally affects the spectral profiles and peaks. However, the sizeable steric hindrance affords an ultrahigh photoluminescence quantum yield (PLQY) of 76.7% for MP-BDP, which significantly surpasses that (41.1%) of P-BDP. Accordingly, MP-BDP exhibits a longer fluorescence lifetime (4.3 ns) than P-BDP (2.8 ns). Quantum chemical calculations indicate that the root-mean-square deviation (RMSQ) value of MP-BDP is 0.05 ?, which is significantly lower than that of P-BDP (RMSD=0.22 ?), thereby indicating that P-BDP undergoes greater conformational changes during the transition from the S0 to S1 states. Additionally, the reorganization energy of P-BDP is 824.2 cm-1, which is much higher than that (659.7 cm-1) of MP-BDP. This suggests that P-BDP with a lower steric hindrance loses more energy through nonradiative transition channels, thus resulting in a lower fluorescence quantum yield. Furthermore, the results of femtosecond transient absorption spectroscopy confirm that the greater steric hindrance in MP-BDP significantly reduces the vibrational relaxation rate, decreases nonradiative energy loss, and enhances fluorescence performance. More importantly, the larger steric groups contribute significantly in inhibiting aggregation-induced quenching, thus allowing MP-BDP NPs to maintain a long fluorescence lifetime (4.8 ns) and high fluorescence efficiency (38.1%) in practical applications. This characteristic enables MP-BDP NPs to realize excellent two-photon fluorescence lifetime imaging of zebrafish.
This study systematically investigated the effect of steric hindrance on the excited-state dynamics of BODIPY. The result shows that steric hindrance improves the fluorescence efficiency and prolongs the fluorescence lifetime of BODIPY by inhibiting nonradiative energy loss through vibrational relaxation. Steady-state spectroscopy and theoretical calculations show that different meso-substituents exhibit similar configurations in the ground state and do not significantly affect the absorption/emission spectra. However, in the excited state, the substituents in MP-BDP exhibit greater steric hindrance. The results of femtosecond transient absorption spectroscopy show that the greater steric effects in MP-BDP effectively suppress the nonradiative energy loss from the S1 state via vibrational relaxation, thereby extending the fluorescence lifetime and enhancing the fluorescence efficiency. Additionally, the larger steric groups effectively inhibit aggregation-induced quenching, thus affording successful two-photon fluorescence lifetime imaging at the in-vivo level. This study provides deeper understanding regarding the effect of molecular conformation on fluorescence properties from the perspective of excited-state dynamics, thus providing important theoretical insights for designing efficient BODIPY fluorescent dyes and offering new approaches for developing new two-photon FLIM agents.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107302 (2024)
The stable supply of cerebral blood flow (CBF) is crucial for maintaining the normal physiological function of the brain. This regulation relies on complex physiological mechanisms involving the collaboration between the basilar arteries and the microvascular network that penetrates the brain parenchyma. Although advances in non-invasive detection technology have provided new opportunities for cerebrovascular research, relevant studies remain limited. Numerous studies are confined to single-scale analyses of blood flow changes, lacking extensive investigations into the interaction between large arteries and microvascular blood flow. Additionally, there are relatively few comprehensive evaluations of cerebrovascular regulatory mechanisms and neurovascular coupling in response to dynamic stimulation conditions. Therefore, an in-depth investigation of the interaction between large arteries and microvascular blood flow patterns not only enhances understanding of the mechanisms governing cerebral blood flow regulation but also provides a critical basis for the early diagnosis, prevention, and treatment of cerebrovascular diseases. The aim of this study was to systematically assess the interaction of cerebral macrovascular and microvascular blood flow patterns under different physiological tasks by integrating transcranial Doppler ultrasound (TCD) and diffuse correlation spectroscopy (DCS) techniques. The outcomes of this study will contribute to current brain research, offering new concepts and methods to improve medical interventions for cerebrovascular diseases.
A total of 16 young, healthy participants (8 women and 8 men, ages 18 to 25) were recruited for this study. They were instructed to perform four tasks: the verbal fluency task (VFT), high-level cognitive task (HCT), voluntary breath-holding (VBH), and postural change task (PCT). Simultaneous changes in microvascular cerebral blood flow (Fcb) and middle cerebral artery blood flow velocity (Vm) during each task were measured using DCS and TCD, respectively. The experimental design adhered strictly to international ethical principles to ensure the safety of the subjects. Precise placement of the TCD probe and DCS optical probe on the subjects’ frontal heads allowed for real-time monitoring of blood flow dynamics in specific brain regions. During data analysis, slope (S), D-index (Dindex), and response time (TR) were defined, and a wide range of statistical variables, including analysis of variance (ANOVA) and Pearson correlation analysis, were utilized to comprehensively assess the correlation and asynchrony between Vm and Fcb across different tasks. The integration of multiple technical approaches ensures the accuracy and reliability of the research findings, providing robust data to reveal the mechanisms underlying cerebral blood flow regulation.
The results indicate that Fcb outperforms Vm in both initial response and response amplitude for all four tasks. Specifically, the initial slope and D-index of Fcb are higher, demonstrating
This study investigated the influences of different physiological paradigms (active and passive stimulation) on cerebral hemodynamics at two scales: macrovascular (measured by TCD) and microvascular (measured by DCS). During the physiological tests, the VFT and HCT primarily triggered neuronal excitation at varying cognitive intensities and explored the rapid response of cerebral blood flow. In contrast, the VBH and PCT tasks challenged the brain’s ability to regulate itself automatically under hypoxic conditions. The asynchrony and correlation between the two measurements suggest that changes in major artery blood velocity support the maintenance of cerebral circulation at the microvascular level. The combination of TCD and DCS provides a comprehensive assessment of neurovascular coupling, demonstrating significant potential for diagnosing cerebrovascular diseases and psychiatric disorders. This study emphasizes the importance of examining hemodynamics in both large and microvessels of the brain to achieve a thorough understanding of neurovascular functions.
.- Publication Date: Nov. 04, 2024
- Vol. 51, Issue 21, 2107303 (2024)
Ultrasound has been widely used in various clinical fields as a safe, low-cost, portable, and fast imaging technology. Currently, piezoelectric transducers are the primary means of generating ultrasound. However, the limited bandwidth of the ultrasound signals generated by the piezoelectric effect restricts high-resolution ultrasound applications. Furthermore, miniaturizing piezoelectric transducers is challenging, and a reduction in their size leads to decreased sensitivity. However, high-precision ultrasound is indispensable for biomedical applications. In the photoacoustic process, light is converted into sound waves. Photoacoustic technology is rapidly being developed for photoacoustic imaging, diagnosis, and sensing in the life sciences. Recently, laser-generated ultrasound technology has emerged as a novel technique distinct from traditional piezoelectric methods. The transducer usually consists of a light absorber and surrounding medium with a high coefficient of thermal expansion. The light absorber can effectively convert light energy into heat energy, and the surrounding medium allows it to produce high-intensity ultrasound. This is a relatively simple method of generating a broadband high-frequency ultrasound signal using a short-pulse laser. This technology combines the excellent imaging depth of ultrasound with the resolution of an optical method, enabling a higher sound pressure and broader bandwidth. It also offers high-resolution ultrasound imaging and therapeutic capabilities with minimal electromagnetic interference. Laser-generated ultrasound transducers have significant advantages in relation to miniaturization, providing a new perspective for the further development of multifunctional ultrasound technology and a novel approach for clinical precision diagnosis and treatment in the future.
Progress The composition of an optical ultrasound transducer has transitioned from a single nanoparticle coating or a single metal layer to a composite of different nanoparticles and polymers. With the adoption of composite materials with high optical absorption and expansion coefficients, there are various designs for optical ultrasound transducers. The basic performance of a transducer is determined by the nanoparticles utilized (candle soot, carbon black, graphene, carbon nanotubes, gold nanoparticles), while the rich variety of transducer structures (flat, concave, fiber) provides flexible solutions for different biomedical applications (Figs. 1 and 2). Short laser pulses are utilized to provide a broad acoustic bandwidth, resulting in higher resolution, less tissue impact, and minimum electromagnetic interference. Recent studies have used optical ultrasound transducers in new high-resolution functional ultrasound applications. The flexibility of the photoacoustic film allows it to be integrated on the surface of the fiber or within the probe (Figs. 4 and 5) to achieve a compact optical ultrasound transducer that can be incorporated into minimally invasive surgical devices used in a variety of microscale biomedical imaging applications (Figs. 6 and 7). Simultaneously, because of the self-focusing effect of the concave transducer and high photoacoustic conversion efficiency of the material, an optical transducer can provide a concentrated sound pressure, tight focus, and higher resolution (Fig. 3). The high-amplitude ultrasound produced by an optical transducer has been utilized in cavitation therapy, thrombolysis, and other functional areas (Figs. 8?10). Moreover, the optical resolution provided by an optical ultrasound transducer offers higher precision for neuromodulation (Fig. 12). In summary, optical ultrasound transducers have been extensively applied in the biomedical field because of their unique advantages.
Conclusions and Prospects Optical ultrasound transducers are an emerging device that complements piezoelectric transducers. Because of their excellent material properties, optical ultrasound transducers have advantages such as a wide bandwidth, high resolution, electromagnetic immunity, flexibility, and ease of miniaturization. These features make the technology more suitable for applications requiring high-pressure ultrasound, high resolution, and micro-scale precision. As advancements are made in material systems and processing in the future, optical ultrasound transducers will drive progress in clinical applications such as diagnostics and imaging, as well as in precision machining. The performance and functionality of optical ultrasound transducers will further improve in the future, offering groundbreaking opportunities for high-resolution imaging and detection, precise localization, image-guided surgery and therapy, interventional procedures, and micro-scale treatment and modulation. Further research by scientists on optical ultrasound transducers is important for their utilization in clinical technologies.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107201 (2024)
Magnetic-induction hyperthermia, also known as magnetic hyperthermia, is an emerging physical-therapy modality for treating tumors clinically. It utilizes magnetic nanomaterials to generate heat under an alternating magnetic field (AMF), thereby triggering molecular events that selectively destroy tumor cells. Magnetic hyperthermia offers several advantages over conventional treatments such as chemotherapy and radiotherapy, including its minimal invasiveness, remote controllability, potential for repeatable treatments, and ability to induce anti-tumor immunity. These characteristics highlight the significant potential of magnetic hyperthermia in clinical tumor treatment. Over the recent decades, continuous effort has been expended to enhance the design of AMF generators, with emphasis on the integration of inverter and power-amplification technologies. However, comprehensive summaries regarding the development of AMF generators are insufficient. Therefore, an overview of currently used AMF generators is necessary to encourage the development of magnetic-hyperthermia devices for clinical applications.
Progress This paper elucidates the operating principles, coil types, and strategies for optimizing the AMF characteristic parameters—such as frequency, field strength, and uniformity—of magnetic-hyperthermia generators. We review relevant magnetic-hyperthermia devices and summarize their advantages and disadvantages. AMF generators generate two types of magnetic fields, i.e., coil and core types. The coil type uses various conductor configurations, such as spiral tubes, flat coils, and Helmholtz coils. By contrast, the core type involves a conductor wrapped around a magnetically conductive medium, such as ferrite, which transfers magnetic field energy and creates an AMF concentrated in the ferrite gap. Additionally, we introduce two magnetic-hyperthermia systems equipped with AMF focusing and waveform-transformation capabilities. Previous studies utilized three hollow spiral tubes as output terminals to construct a magnetic-hyperthermia system equipped with AMF focusing (Fig. 2). These systems address the problem of dispersed action ranges. Systems with waveform-transformation capabilities can be used to investigate the effects of different magnetic-field waveforms on the magneto-thermal efficiency of magnetic nanomaterials. Furthermore, we discuss a research platform that integrates optical testing instruments, i.e., a confocal laser scanning microscope and spectrometer, with a magnetic-hyperthermia device. Researchers have designed small ferrite coils with narrow gaps for confocal laser scanning microscopes to realize a research platform (Fig. 3). Similarly, Helmholtz-like coils have been used in conjunction with a spectrometer to create platforms (Fig. 4). This integration provides a unique tool for performing comprehensive studies on the magnetic-hyperthermia effect at the molecular/cellular level. Additionally, this study reviews existing clinical magnetic-hyperthermia devices and discusses their potential clinical applications. These devices include the first prototype developed by Jordan’s group in Germany, the NFH?300F commercial magnetic-hyperthermia equipment developed by Gneveckow’s group in Germany, and a third-generation equipment developed by Tang Jintian’s group in China. Continuous technological advancements and optimizations are expected to enhance the therapeutic efficacy and prognosis of tumor physical therapy, thus rendering magnetic hyperthermia a crucial tool in future cancer treatments.
Conclusions and Prospects Magnetic-hyperthermia devices are pivotal in advancing research and enhancing oncology treatments. Although most current devices are primarily used in cellular and animal experiments, their broader clinical application requires emphasis on several key points. First, magnetic-hyperthermia devices must be able to generate therapeutic magnetic fields suitable for the human body. Hence, low-loss, high-power coils with a wide spatial range must be developed. Second, to generate wide-ranging, high-intensity, and high-frequency magnetic fields, inverter-circuit components must withstand high voltages and currents. Accurate control systems are required to reduce power loss. Third, as equipment power increases, better cooling systems are necessitated for stability and safety improvement. Additionally, magnetic field-focused devices should minimize heat damage to normal tissues, thereby reducing side effects. Developing new generations of magnetic-hyperthermia devices will advance scientific research and provide a foundation for widespread clinical applications, thus potentially offering optimism for cancer treatment.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107202 (2024)
Studies show that breast cancer ranks first among all cancers affecting women in China. Additionally, its mortality rate is high and continues to increase. The five-year relative survival rate of patients with breast cancer who are detected at an early stage is ? 90%, whereas it is only 20% for those detected at an advanced stage. Therefore, early and effective diagnosis of breast cancer is crucial to patient recovery. Invasive ductal carcinoma is one of the most common types of breast cancer, and sclerosing adenopathy is a benign breast disease. These two diseases are similar both visually and microscopically and are susceptible to diagnostic errors. Immunohistochemical staining is typically required to identify these markers. It is a complicated and costly technique that can yield false-positive or false-negative results. Therefore, new methods for identifying sclerosing adenopathy and early invasive ductal carcinoma must be devised to reduce the misdiagnosis rates of both diseases. Polarimetric imaging can reflect the intricate microstructure of biological tissues and is promising for detecting biological tissues. Therefore, differential-tissue analysis for detecting sclerosing adenopathy and early invasive ductal carcinoma using polarization imaging methods is worth investigating.
Polarization imaging experiments were performed on sclerosing adenopathy and early invasive ductal carcinoma tissues. Subsequently, four vector parameters, i.e., the delay vector (R), dichroic vector (D), polarization vector (P), and scattering receding vector (
In this study, Mueller-matrix parameters were extracted via MMPD and MMT methods using breast sclerosing adenopathy and early invasive ductal carcinoma tissues, which are easily misdiagnosed in clinical practice. The effect of the Mueller-matrix parameters in characterizing the difference between the two diseases in two different structures, i.e., fibrotic mesenchyme and nucleus, was investigated based on the distribution range using the minimum enclosing sphere. The result shows that in the fibrotic interstitial region, the MMPD parameter R (Fig. 3) and the combinations of three-parameter (Fig. 4), i.e., R-θ-ψ, δ-θ-ψ, and R-δ-ψ, resulted in a better characterization of the difference between the two diseases. The relative radius difference of the smallest encircling sphere for the four abovementioned characterization parameters was 68.77%, 61.33%, 61.09%, and 61.04%, respectively (Table 2). This may be because different mechanisms and degrees of fibrotic mesenchymal proliferation occur in both diseases, to which phase-delay-related parameters are extremely sensitive. In the nucleus region, the MMPD parameter
Investigations into the differences between sclerosing adenopathy and early invasive ductal carcinoma are important in clinical applications. In this study, the variability of the two diseases was investigated using a biotissue Mueller-matrix imaging system. The results show that in the fibrotic mesenchymal region, the parameters related to the fundamental matrix of phase delays obtained using the MMPD method are the best for the differential characterization of sclerosing adenopathy and early invasive ductal carcinoma. They include the phase delay vector R; three-parameter combinations of four scalar parameters (the four parameters are the total phase delay values
For both diseases, the Mueller-matrix parameters yielded better difference characterization in the fibrotic mesenchymal region than in the nucleus region. This study describes the development of clinical discriminators for both diseases. Only the differences in the distribution ranges of the Mueller-matrix parameters were examined, and only one variable, i.e., sample structure, was controlled. In future studies, the effect of the parameter values and variables such as the thickness of the sample slices shall be investigated.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107203 (2024)
This study investigates a novel photothermal composite hydrogel (PGS-Cu/PTCP) derived from cuttlebone β-tricalcium phosphate, which is responsive to near-infrared (NIR) laser for oral photothermal therapy (PTT). This study can potentially contribute to enhancements in the safety and efficacy of bone-defect reconstruction materials via the incorporation of photothermal and antibacterial properties.
A PGS-Cu/PTCP hydrogel was synthesized using cuttlebone β-tricalcium phosphate, polyvinyl alcohol, gelatin, sodium alginate, copper ions, and polydopamine. The porosity, mechanical properties, photothermal antibacterial activity, and biocompatibility of the hydrogel were evaluated. Photothermal performance was assessed by irradiating the hydrogel samples with an 808 nm NIR laser at different power settings, and the temperature changes were recorded using an infrared thermal imaging camera. Cytotoxicity tests were conducted using L929 fibroblast cells, and antibacterial performance was tested against Escherichia coli and Staphylococcus aureus.
The PGS-Cu/PTCP hydrogel exhibits excellent porosity, which is beneficial for nutrient supply and waste removal in bone-tissue engineering. Under 808 nm NIR laser irradiation at 1.05 W, the hydrogel's temperature reached 49.4 ℃ within 420 s, thus indicating high photothermal conversion efficiency (Fig. 3). The hydrogel shows low cytotoxicity and >90% cell viability for L929 cells (Fig. 2). Antibacterial tests showed significantly reduced survival rates of Escherichia coli and Staphylococcus aureus upon NIR irradiation, which differ considerably from those of non-irradiated samples (Fig. 5). The combination of copper ions and polydopamine in the hydrogel contributed to its superior antibacterial properties, which is likely owing to the disruption of bacterial membranes and enhanced photothermal effects.
The novel PGS-Cu/PTCP hydrogel efficiently converts NIR light into thermal energy, thus providing precise in-vivo photothermal antibacterial activity. This material significantly enhances the safety of oral bone-repair applications, thereby rendering it a promising candidate for future clinical use in maxillofacial-bone reconstruction.
.- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2107204 (2024)
- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2116001 (2024)
- Publication Date: Oct. 31, 2024
- Vol. 51, Issue 21, 2116002 (2024)