• Advanced Photonics
  • Vol. 6, Issue 6, 064001 (2024)
Dana Hassan1,2, Jesús Domínguez1,3, Benjamin Midtvedt1, Henrik Klein Moberg4..., Jesús Pineda1, Christoph Langhammer4, Giovanni Volpe1, Antoni Homs Corbera2 and Caroline B. Adiels1,*|Show fewer author(s)
Author Affiliations
  • 1University of Gothenburg, Department of Physics, Gothenburg, Sweden
  • 2CherryBiotech, Research and Development Unit, Montreuil, France
  • 3Elvesys – Microfluidics Innovation Center, Elvesys, Paris, France
  • 4Chalmers University of Technology, Department of Physics, Gothenburg, Sweden
  • show less
    DOI: 10.1117/1.AP.6.6.064001 Cite this Article Set citation alerts
    Dana Hassan, Jesús Domínguez, Benjamin Midtvedt, Henrik Klein Moberg, Jesús Pineda, Christoph Langhammer, Giovanni Volpe, Antoni Homs Corbera, Caroline B. Adiels, "Cross-modality transformations in biological microscopy enabled by deep learning," Adv. Photon. 6, 064001 (2024) Copy Citation Text show less
    References

    [1] D. Murphy, M. Davidson. Fundamentals of Light Microscopy and Electronic Imaging(2012).

    [2] K. Suvarna, C. Layton, J. Bancroft. Bancroft’s Theory and Practice of Histological Techniques E-Book(2012).

    [3] J. W. Lichtman, J.-A. Conchello. Fluorescence microscopy. Nat. Methods, 2, 910-919(2005).

    [4] I. Johnson. Molecular probes handbook: a guide to fluorescent probes and labeling technologies(2010).

    [5] H. Sahoo. Fluorescent labeling techniques in biomolecules: a flashback. RSC Adv., 2, 7017-7029(2012).

    [6] S. Shashkova, M. Leake. Single-molecule fluorescence microscopy review: shedding new light on old problems. Biosci. Rep., 37, BSR20170031(2017).

    [7] M. Gurcan et al. Histopathological image analysis: a review. IEEE Rev. Biomed. Eng., 2, 147-171(2009).

    [8] F. Helmchen, W. Denk. Deep tissue two-photon microscopy. Nat. Methods, 2, 932-940(2005).

    [9] Y.-F. He et al. Deep-learning driven, high-precision plasmonic scattering interferometry for single-particle identification. ACS Nano, 18, 9704-9712(2024).

    [10] E. Nehme et al. DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning. Nat. Methods, 17, 734-740(2020).

    [11] J. Ferreira, L. Groc. Surface Glutamate Receptor Nanoscale Organization with Super-Resolution Microscopy (dSTORM), 35-52(2024).

    [12] M. D. Wilkinson et al. The fair guiding principles for scientific data management and stewardship. Sci. Data, 3, 160018(2016).

    [13] Y. Rivenson et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng., 3, 466-477(2019).

    [14] S. Shafi, A. V. Parwani. Artificial intelligence in diagnostic pathology. Diagn. Pathol., 18, 109(2023).

    [15] E. M. Christiansen et al. In silico labeling: predicting fluorescent labels in unlabeled images. Cell, 173, 792-803(2018).

    [16] H. Wang et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods, 16, 103-110(2019).

    [17] X. Cao et al. Deep learning based inter-modality image registration supervised by intra-modality similarity. Lecture Notes in Computer Science, 55-63(2018).

    [18] L. Latonen et al. Virtual staining for histology by deep learning. Trends Biotechnol., 42, 1177-1191(2024).

    [19] J. Johnson, A. Alahi, L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. Lecture Notes in Computer Science, 694-711(2016).

    [20] C. Ledig et al. Photo-realistic single image super-resolution using a generative adversarial network, 15640-15649(2017).

    [21] K. Zhang et al. Negative-aware attention framework for image-text matching(2022).

    [22] T. S. Gurina, L. Simms. Histology, staining(2023).

    [23] C. L. Chen et al. Deep learning in label-free cell classification. Sci. Rep., 6, 21471(2016).

    [24] M. Hermsen et al. Deep learning-based histopathologic assessment of kidney tissue. J. Am. Soc. Nephrol., 30, 1968-1979(2019).

    [25] B. Bai et al. Label-free virtual HER2 immunohistochemical staining of breast tissue using deep learning. BME Frontiers, 2022, 9786242(2022).

    [26] S. Soltani et al. Prostate cancer histopathology using label-free multispectral deep-UV microscopy quantifies phenotypes of tumor aggressiveness and enables multiple diagnostic virtual stains. Sci. Rep., 12, 9329(2022).

    [27] N. Pillar, A. Ozcan. Virtual tissue staining in pathology using machine learning. Expert Rev. Molecular Diagnostics, 22, 987-989(2022).

    [28] D. S. Richardson, J. W. Lichtman. Clarifying tissue clearing. Cell, 162, 246-257(2015).

    [29] X. Wang et al. Single-shot isotropic differential interference contrast microscopy. Nat. Commun., 14, 2063(2023).

    [30] F. Pan et al. Accurate detection and instance segmentation of unstained living adherent cells in differential interference contrast images. Comput. Biol. Med., 182, 109151(2024).

    [31] M. Z. Hoque et al. Stain normalization methods for histopathology image analysis: a comprehensive review and experimental comparison. Inf. Fusion, 102, 101997(2024).

    [32] A. Tomczak et al. Multi-task multi-domain learning for digital staining and classification of leukocytes. IEEE Trans. Med. Imaging, 40, 2897-2910(2021).

    [33] J. Park et al. Artificial intelligence-enabled quantitative phase imaging methods for life sciences. Nat. Methods, 20, 1645-1660(2023).

    [34] E. Breznik et al. Cross-modality sub-image retrieval using contrastive multimodal image representations. Sci. Rep., 14, 18798(2024).

    [35] A. Lahiani et al. Virtualization of tissue staining in digital pathology using an unsupervised deep learning approach. Lecture Notes in Comput. Sci., 47-55(2019).

    [36] G. Zhang et al. Image-to-images translation for multiple virtual histological staining of unlabeled human carotid atherosclerotic tissue. Molecular Imaging Biol., 24, 31-41(2022).

    [37] J. Li et al. Biopsy-free in vivo virtual histology of skin using deep learning. Light Sci. Appl., 10, 233(2021).

    [38] T. M. Abraham et al. Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging and virtual hematoxylin and eosin staining. Optica, 10, 1605-1618(2023).

    [39] A. Rana et al. Use of deep learning to develop and analyze computational hematoxylin and eosin staining of prostate core biopsy images for tumor diagnosis. JAMA Network Open, 3, e205111(2020).

    [40] D. Li et al. Deep learning for virtual histological staining of bright-field microscopic images of unlabeled carotid artery tissue. Molecular Imaging Biol., 22, 1301-1309(2020).

    [41] E. A. Burlingame et al. Shift: speedy histological-to-immunofluorescent translation of a tumor signature enabled by deep learning. Sci. Rep., 10, 17507(2020).

    [42] B. Shen et al. Deep learning autofluorescence-harmonic microscopy. Light Sci. Appl., 11, 76(2022).

    [43] Y. Zhang et al. Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light Sci. Appl., 9, 78(2020).

    [44] K. de Haan et al. Deep learning-based transformation of h&e stained tissues into special stains. Nat. Commun., 12, 4884(2021).

    [45] Y. Rivenson et al. Phasestain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light Sci. Appl., 8, 23(2019).

    [46] M. Boktor et al. Virtual histological staining of label-free total absorption photoacoustic remote sensing (TA-PARS). Sci. Rep., 12, 10296(2022).

    [47] J. J. Levy et al. A large-scale internal validation study of unsupervised virtual trichrome staining technologies on nonalcoholic steatohepatitis liver biopsies. Mod. Pathol., 34, 808-822(2021).

    [48] Y. Hong et al. Deep learning-based virtual cytokeratin staining of gastric carcinomas to measure tumor–stroma ratio. Sci. Rep., 11, 19255(2021).

    [49] Q. Dou et al. PNP-Adanet: plug-and-play adversarial domain adaptation network at unpaired cross-modality cardiac segmentation. IEEE Access, 7, 99065-99076(2019).

    [50] J. Y. Zhu et al. Unpaired image-to-image translation using cycle-consistent adversarial networks, 2242-2251(2017).

    [51] M. Kawai et al. Virtual multi-staining in a single-section view for renal pathology using generative adversarial networks. Comput. Biol. Med., 182, 109149(2024).

    [52] Y. Rivenson et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng., 3, 466-477(2019).

    [53] S. Koivukoski et al. Unstained tissue imaging and virtual hematoxylin and eosin staining of histologic whole slide images. Lab. Invest., 103, 100070(2023).

    [54] S. Helgadottir et al. Extracting quantitative biological information from bright-field cell images using deep learning. Biophys. Rev., 2, 031401(2021).

    [55] X. Li et al. Unsupervised content-preserving transformation for optical microscopy. Light Sci. Appl., 10, 44(2021).

    [56] E. Nehme et al. Deep-storm: super-resolution single-molecule microscopy by deep learning. Optica, 5, 458-464(2018).

    [57] P. Zhang et al. Analyzing complex single-molecule emission patterns with deep learning. Nat. Methods, 15, 913-916(2018).

    [58] A. Speiser et al. Deep learning enables fast and dense single-molecule localization with high accuracy. Nat. Methods, 18, 1082-1090(2021).

    [59] L. Möckl et al. Accurate and rapid background estimation in single-molecule localization microscopy using the deep neural network BGnet. Proc. Natl. Acad. Sci., 117, 60-67(2019).

    [60] W. Ouyang et al. Deep learning massively accelerates super-resolution localization microscopy. Nat. Biotechnol., 36, 460-468(2018).

    [61] R. Sanyal, D. Kar, R. Sarkar. Carcinoma type classification from high-resolution breast microscopy images using a hybrid ensemble of deep convolutional features and gradient boosting trees classifiers. IEEE/ACM Trans. Comput. Biol. Bioinf., 19, 2124-2136(2021).

    [62] D. Zhang et al. Cross-modality deep feature learning for brain tumor segmentation. Pattern Recognit., 110, 107562(2021).

    [63] Z. Wang et al. Real-time volumetric reconstruction of biological dynamics with light-field microscopy and deep learning. Nat. Methods, 18, 551-556(2021).

    [64] M. S. Durkee et al. Artificial intelligence and cellular segmentation in tissue microscopy images. Am. J. Pathol., 191, 1693-1701(2021).

    [65] Q. Dou et al. Unsupervised cross-modality domain adaptation of convnets for biomedical image segmentations with adversarial loss, 691-697(2018).

    [66] Q. Li et al. A cross-modality learning approach for vessel segmentation in retinal images. IEEE Trans. Med. Imaging, 35, 109-118(2016).

    [67] S. Klein et al. Deep learning predicts HPV association in oropharyngeal squamous cell carcinomas and identifies patients with a favorable prognosis using regular H&E stains. Clin. Cancer Res., 27, 1131-1138(2021).

    [68] W. Xie et al. Prostate cancer risk stratification via nondestructive 3D pathology with deep learning–assisted gland analysis. Cancer Res., 82, 334-345(2022).

    [69] H. Lee et al. Revisiting the use of structural similarity index in Hi-C. Nat. Genet., 55, 2049-2052(2023).

    [70] D. Gu et al. An artificial-intelligence-based age-specific template construction framework for brain structural analysis using magnetic resonance images. Hum. Brain Mapp., 44, 861-875(2023).

    [71] Z. Yu et al. Need for objective task-based evaluation of deep learning-based denoising methods: a study in the context of myocardial perfusion spect. Med. Phys., 50, 4122-4137(2023).

    [72] M. Dohmen et al. Similarity metrics for MR image-to-image translation(2024).

    [73] J. Ke et al. Artifact detection and restoration in histology images with stain-style and structural preservation. IEEE Trans. Med. Imaging, 42, 3487-3500(2023).

    [74] R. Houhou et al. Comparison of denoising tools for the reconstruction of nonlinear multimodal images. Biomed. Opt. Express, 14, 3259-3278(2023).

    [75] G. Litjens et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep., 6, 26286(2016).

    [76] M. Luella, B. Paul, A. Javad. Generative AI in medical imaging and its application in low dose computed tomography (CT) image denoising. Applications of Generative AI, 387-401(2024).

    [77] Y. Choi et al. StarGAN: unified generative adversarial networks for multi-domain image-to-image translation, 8789-8797(2018).

    [78] J. Vasiljević et al. HistostarGAN: a unified approach to stain normalisation, stain transfer and stain invariant segmentation in renal histopathology. Knowl.-Based Syst., 277, 110780(2023).

    [79] C. L. Srinidhi, O. Ciga, A. L. Martel. Deep neural network models for computational histopathology: a survey. Med. Image Anal., 67, 101813(2021).

    [80] J. van der Laak, G. Litjens, F. Ciompi. Deep learning in histopathology: the path to the clinic. Nat. Med., 27, 775-784(2021).

    [81] A. Echle et al. Deep learning in cancer pathology: a new generation of clinical biomarkers. Br. J. Cancer, 124, 686-696(2020).

    [82] S. Banerji, S. Mitra. Deep learning in histopathology: a review. WIREs Data Min. Knowl. Discovery, 12, e1439(2022).

    [83] J. Xu et al. Deep Learning for Histopathological Image Analysis: Towards Computerized Diagnosis on Cancers, 73-95(2017).

    [84] Y. He et al. PST-Diff: achieving high-consistency stain transfer by diffusion models with pathological and structural constraints. IEEE Trans. Med. Imaging, 43, 3634-3647(2024).

    [85] M. J. Fanous, G. Popescu. GANscan: continuous scanning microscopy using deep learning deblurring. Light Sci. Appl., 11, 265(2022).

    [86] Y. Rivenson, A. Ozcan. Deep learning accelerates whole slide imaging for next-generation digital pathology applications. Light Sci. Appl., 11, 300(2022).

    [87] A. Su et al. A deep learning model for molecular label transfer that enables cancer cell identification from histopathology images. NPJ Precis. Oncol., 6, 14(2022).

    [88] S. M. Hickey et al. Fluorescence microscopy: an outline of hardware, biological handling, and fluorophore considerations. Cells, 11, 35(2022).

    [89] P. A. Moghadam et al. A morphology focused diffusion probabilistic model for synthesis of histopathology images, 1999-2008(2023).

    [90] A. Greenspan, Y. Shen, J. Keet?al.. StainDiff: transfer stain styles of histology images with denoising diffusion probabilistic models and self-ensemble. Med. Image Computing and Computer Assisted Intervention, 549-559(2023).

    [91] T. Kataria, B. Knudsen, S. Y. Elhabian. StainDiffuser: multitask dual diffusion model for virtual staining(2024).

    [92] S. Dubey, X. Xu et al. VIMS: virtual immunohistochemistry multiplex staining via text-to-stain diffusion trained on uniplex stains. Mach. Learn. Med. Imaging, 143-155(2024).

    [93] T. M. Abraham, R. Levenson. A comparison of diffusion models and CycleGANs for virtual staining of slide-free microscopy images, 1-6(2023).

    [94] R. Rizzuto et al. Chimeric green fluorescent protein as a tool for visualizing subcellular organelles in living cells. Curr. Biol., 5, 635-642(1995).

    [95] O. Kepp et al. Cell death assays for drug discovery. Nat. Rev. Drug Disc., 10, 221-237(2011).

    [96] E. Moen et al. Deep learning for cellular image analysis. Nat. Methods, 16, 1233-1246(2019).

    [97] V. Lulevich et al. Cell tracing dyes significantly change single cell mechanics. J. Phys. Chem. B, 113, 6511-6519(2009).

    [98] A. J. Hobro, N. I. Smith. An evaluation of fixation methods: spatial and compositional cellular changes observed by Raman imaging. Vib. Spectrosc., 91, 31-45(2017).

    [99] J. Hira et al. From differential stains to next generation physiology: chemical probes to visualize bacterial cell structure and physiology. Molecules, 25, 4949(2020).

    [100] E. C. Jensen. Overview of live-cell imaging: requirements and methods used. Anat. Rec., 296, 1-8(2013).

    [101] S. N. Chandrasekaran et al. Image-based profiling for drug discovery: due for a machine-learning upgrade?. Nat. Rev. Drug Disc., 20, 145-159(2021).

    [102] T. C. Nguyen et al. Virtual organelle self-coding for fluorescence imaging via adversarial learning. J. Biomed. Opt., 25, 096009(2020).

    [103] M. E. Kandel et al. Multiscale assay of unlabeled neurite dynamics using phase imaging with computational specificity. ACS Sens., 6, 1864-1874(2021).

    [104] Y. N. Nygate et al. Holographic virtual staining of individual biological cells. Proc. Natl. Acad. Sci., 117, 9223-9231(2020).

    [105] C. Ounkomol et al. Label-free prediction of three-dimensional fluorescence images from transmitted-light microscopy. Nat. Methods, 15, 917-920(2018).

    [106] S. Cheng et al. Single-cell cytometry via multiplexed fluorescence prediction by label-free reflectance microscopy. Sci. Adv., 7, eabe0431(2021).

    [107] W. Song et al. Led array reflectance microscopy for scattering-based multi-contrast imaging. Opt. Lett., 45, 1647-1650(2020).

    [108] C. L. Cooke et al. Physics-enhanced machine learning for virtual fluorescence microscopy, 3803-3813(2021).

    [109] V. Mannam et al. Deep learning-based super-resolution fluorescence microscopy on small datasets. Proc. SPIE, 11650, 116500O(2021).

    [110] L. Xu et al. Deep learning enables stochastic optical reconstruction microscopy-like superresolution image reconstruction from conventional microscopy. iScience, 26, 108145(2023).

    [111] F. Zhao et al. Deep-learning super-resolution light-sheet add-on microscopy (Deep-SLAM) for easy isotropic volumetric imaging of large biological specimens. Opt. Express, 11, 7273-7285(2020).

    [112] R. J. G. van Sloun et al. Super-resolution ultrasound localization microscopy through deep learning. IEEE Trans. Med. Imaging, 40, 829-839(2021).

    [113] X. Zhuang. Nano-imaging with storm. Nat. Photonics, 3, 365-367(2009).

    [114] B. Huang, M. Bates, X. Zhuang. Super-resolution fluorescence microscopy. Annu. Rev. Biochem., 78, 993-1016(2009).

    [115] M. J. Rust, M. Bates, X. Zhuang. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat. Methods, 3, 793-796(2006).

    [116] H. Shroff, H. White, E. Betzig. Photoactivated localization microscopy (PALM) of adhesion complexes. Curr. Protoc. Cell Biol., 41, 4.21.1-4.21.27(2013).

    [117] S. T. Hess, T. P. Girirajan, M. D. Mason. Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. Biophys. J., 91, 4258-4272(2006).

    [118] M. Jung, D. Kim, J. Y. Mun. Direct visualization of actin filaments and actin-binding proteins in neuronal cells. Front. Cell Dev. Biol., 8, 588556(2020).

    [119] E. Wolf. Progress in Optics(2008).

    [120] M. Lelek et al. Single-molecule localization microscopy. Nat. Rev. Methods Primers, 1, 39(2021).

    [121] C. Ledig et al. Photo-realistic single image super-resolution using a generative adversarial network, 105-114(2016).

    [122] J. Grant-Jacob et al. A neural lens for super-resolution biological imaging. J. Phys. Commun., 3, 7(2019).

    [123] C. Chen et al. Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation, 865-872(2019).

    [124] D. Maleki, H. Tizhoosh. LILE: look in-depth before looking elsewhere: a dual attention network using transformers for cross-modal information retrieval in histopathology archives(2022).

    [125] Q. Yang et al. MRI cross-modality image-to-image translation. Sci. Rep., 10, 3753(2020).

    [126] R. Naseem et al. Cross modality guided liver image enhancement of CT using MRI, 46-51(2019).

    [127] C. Dong, D. Fleet et al. Learning a deep convolutional network for image super-resolution, 184-199(2014).

    [128] Y. Zheng et al. A hybrid convolutional neural network for super-resolution reconstruction of MR images. Med. Phys., 47, 3013-3022(2020).

    [129] J. Chun et al. MRI super-resolution reconstruction for MRI-guided adaptive radiotherapy using cascaded deep learning: in the presence of limited training data and unknown translation model. Med. Phys., 46, 4148-4164(2019).

    [130] J. Lv et al. Reconstruction of undersampled radial free-breathing 3D abdominal MRI using stacked convolutional auto-encoders. Med. Phys., 45, 2023-2032(2018).

    [131] H. Li et al. Fast and accurate super-resolution of MR images based on lightweight generative adversarial network. Multimedia Tools Appl., 82, 2465-2487(2023).

    [132] L. Kang et al. Super-resolution method for MR images based on multi-resolution CNN. Biomed. Signal Process. Control, 72, 103372(2022).

    [133] E. Kang, J. Min, J. C. Ye. A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction. Med. Phys., 44, e360-e375(2017).

    [134] C. Dong, J. M. Leibe, C. C. Loy, X. Tang et al. Accelerating the super-resolution convolutional neural network. Computer Vision–ECCV 2016, 391-407(2016).

    [135] Y.-B. Du et al. X-ray image super-resolution reconstruction based on a multiple distillation feedback network. Appl. Intell., 51, 5081-5094(2021).

    [136] Y. Almalioglu et al. EndoL2H: deep super-resolution for capsule endoscopy. IEEE Trans. Med. Imaging, 39, 4297-4309(2020).

    [137] D. Ravì et al. Effective deep learning training for single-image super-resolution in endomicroscopy exploiting video-registration-based reconstruction. Int. J. Comput. Assisted Radiol. Surgery, 13, 917-924(2018).

    [138] M. Srikrishna et al. Deep learning from MRI-derived labels enables automatic brain tissue classification on human brain CT. NeuroImage, 244, 118606(2021).

    [139] M. Haris, G. Shakhnarovich, N. Ukita. Deep back-projection networks for super-resolution, 1664-1673(2018).

    [140] O. Oktay et al. Anatomically constrained neural networks (ACNNS): application to cardiac image enhancement and segmentation. IEEE Trans. Med. Imaging, 37, 384-395(2018).

    [141] M. Srikrishna et al. Comparison of two-dimensional- and three-dimensional-based U-net architectures for brain tissue classification in one-dimensional brain CT. Front. Comput. Neurosci., 15, 785244(2022).

    [142] A. Mani et al. Applying deep learning to accelerated clinical brain magnetic resonance imaging for multiple sclerosis. Front. Neurol., 12, 685276(2021).

    [143] K. Chui et al. An MRI scans-based Alzheimer’s disease detection via convolutional neural network and transfer learning. Diagnostics, 12, 1531(2022).

    [144] J.-Y. Lin, Y.-C. Chang, W. H. Hsu. Efficient and phase-aware video super-resolution for cardiac MRI, 66-76(2020).

    [145] Y. Huang, L. Shao, A. F. Frangi. Simultaneous super-resolution and cross-modality synthesis of 3D medical images using weakly-supervised joint convolutional sparse coding, 5787-5796(2017).

    [146] W.-S. Lai et al. Deep Laplacian pyramid networks for fast and accurate super-resolution, 624-632(2017).

    [147] R. Chen et al. Single-frame deep-learning super-resolution microscopy for intracellular dynamics imaging. Nat. Commun., 14, 2854(2023).

    [148] R. J. Marsh et al. Artifact-free high-density localization microscopy analysis. Nat. Methods, 15, 689-692(2018).

    [149] J. Youn et al. Detection and localization of ultrasound scatterers using convolutional neural networks. IEEE Trans. Med. Imaging, 39, 3855-3867(2020).

    [150] D. Allen et al. High-throughput imaging of CRISPR- and recombinant adeno-associated virus-induced DNA damage response in human hematopoietic stem and progenitor cells. CRISPR J., 5, 80-94(2022).

    [151] N. Boyd et al. DeepLOCO: fast 3D localization microscopy using neural networks. bioRxiv(2018).

    [152] H. Zhou et al. 3D high resolution generative deep-learning network for fluorescence microscopy imaging. Opt. Lett., 45, 1695-1698(2020).

    [153] W. Zhang et al. High-axial-resolution single-molecule localization under dense excitation with a multi-channel deep U-Net. Opt. Lett., 46, 5477-5480(2021).

    [154] P. Zelger et al. Three-dimensional localization microscopy using deep learning. Opt. Express, 26, 33166-33179(2018).

    [155] Y. Li et al. Real-time 3D single-molecule localization using experimental point spread functions. Nat. Methods, 15, 367-369(2018).

    [156] Z. Zhang et al. Machine-learning based spectral classification for spectroscopic single-molecule localization microscopy. Opt. Lett., 44, 5864-5867(2019).

    [157] T. Kim, S. Moon, K. Xu. Information-rich localization microscopy through machine learning. Nat. Commun., 10, 1996(2019).

    [158] E. Hershko et al. Multicolor localization microscopy and point-spread-function engineering by deep learning. Opt. Express, 27, 6158-6183(2019).

    [159] J. Liu et al. Deep learning-enhanced fluorescence microscopy via degeneration decoupling. Opt. Express, 28, 14859-14873(2020).

    [160] J. Li et al. Spatial and temporal super-resolution for fluorescence microscopy by a recurrent neural network. Opt. Express, 29, 15747-15763(2021).

    [161] Y. Ma et al. Cascade neural approximating for few-shot super-resolution photoacoustic angiography. Appl. Phys. Lett., 121, 103701(2022).

    [162] M. Guo et al. Single-shot super-resolution total internal reflection fluorescence microscopy. Nat. Methods, 15, 425-428(2018).

    [163] J. Soh, S. Cho, N. Cho. Meta-transfer learning for zero-shot super-resolution, 3513-3522(2020).

    [164] Z. Burns, Z. Liu. Untrained, physics-informed neural networks for structured illumination microscopy. Opt. Express, 31, 8714-8724(2023).

    [165] H. Sahak et al. Denoising diffusion probabilistic models for robust image super-resolution in the wild(2023).

    [166] S. Gao et al. Implicit diffusion models for continuous super-resolution, 10021-10030(2023).

    [167] J. Ho et al. Cascaded diffusion models for high fidelity image generation. J. Mach. Learn. Res., 23, 1-33(2021).

    [168] R. Rombach et al. High-resolution image synthesis with latent diffusion models, 10674-10685(2021).

    [169] A. Saguy et al. This microtubule does not exist: super-resolution microscopy image generation by a diffusion model, 2400672(2024).

    [170] H. Greenspan, M. Pan et al. DiffuseIR: diffusion models for isotropic reconstruction of 3D microscopic images, 323-332(2023).

    [171] S. Gao et al. GDPR Requirements for Biobanking Activities Across Europe, 10021-10030(2023).

    [172] V. Colcelli et al. GDPR requirements for biobanking activities across europe(2023).

    [173] Health insurance portability and accountability act of 1996 (HIPAA), 104-191(1996).

    [174] (2018).

    [175] Y. Yao, F. Yang. Overcoming personal information protection challenges involving real-world data to support public health efforts in China. Front. Public Health, 11, 1265050(2023).

    [176] D. Dhingra, A. Dabas. Global strategy on digital health. Indian Pediatrics, 57, 356-358(2020).

    [177] The protection of personal data in health information systems–principles and processes for public health(2020).

    [178] D. Jain. Regulation of digital healthcare in India: ethical and legal challenges. Healthcare, 11, 911(2023).

    [179] D. B. Larson et al. Ethics of using and sharing clinical imaging data for artificial intelligence: a proposed framework. Radiology, 295, 675-682(2020).

    [180] A. S. Pillai. Utilizing deep learning in medical image analysis for enhanced diagnostic accuracy and patient care: challenges, opportunities, and ethical implications. J. Deep Learn. Genomic Data Anal., 1, 1-17(2021).

    [181] R. Bouderhem. Shaping the future of AI in healthcare through ethics and governance. Humanities Soc. Sci. Commun., 11, 416(2024).

    [182] N. Forgó et al. Big data, AI and health data: between national, european, and international legal frameworks. Legal Challenges in the New Digital Age, 358-394(2023).

    [183] S. T. Padmapriya, S. Parthasarathy. Ethical data collection for medical image analysis: a structured approach. Asian Bioethics Rev., 16, 95-108(2024).

    [184] T. Willem et al. Risks and benefits of dermatological machine learning health care applications: an overview and ethical analysis. J. Eur. Acad. Dermatol. Venereol., 36, 1660-1668(2022).

    [185] M. Jeyaraman et al. Unraveling the ethical enigma: artificial intelligence in healthcare. Cureus, 15, e43262(2023).

    [186] K. Grünberg et al. Ethical and privacy aspects of using medical image data. Cloud-Based Benchmarking of Medical Image Analysis, 33-43(2017).

    [187] Y. Li et al. Virtual histological staining of unlabeled autopsy tissue. Nat. Commun., 15, 1684(2024).

    [188] H. Kumar, P. Kim. Artificial intelligence in fusion protein three-dimensional structure prediction: review and perspective. Clin. Transl. Med., 14, e1789(2024).

    [189] J. Zhang et al. Ai co-pilot bronchoscope robot. Nat. Commun., 15, 241(2024).

    [190] A. A. Banaeiyan et al. Design and fabrication of a scalable liver-lobule-on-a-chip microphysiological platform. Biofabrication, 9, 15014(2017).

    Dana Hassan, Jesús Domínguez, Benjamin Midtvedt, Henrik Klein Moberg, Jesús Pineda, Christoph Langhammer, Giovanni Volpe, Antoni Homs Corbera, Caroline B. Adiels, "Cross-modality transformations in biological microscopy enabled by deep learning," Adv. Photon. 6, 064001 (2024)
    Download Citation