A promising and controllable way of inertial confinement fusion (ICF) is to focus high-power lasers to compress and heat a fuel capsule (target) positioned at the center of a vacuum target chamber to achieve fusion ignition. In-situ inspection of laser-induced damage (LID) on optics plays an important role for these high-power laser systems^[1], because most large aperture optics within the beamline will be exposed to the high-energy laser pulse. Especially, in the final integrated optics module (IOM), the optics have higher risk to be damaged. The highlight of in-situ inspection is that it can detect and track the damage sites online^[2] and then give an alarm when the LID sites grow to the point at which the optic should be removed to avoid destroying the optic or damaging downstream optics^[3].

We had developed an in-situ final optics damage inspection (FODI) system^[4] which is an optical telescope designed to be inserted to the center of the target chamber after a laser shot (Fig. 1). From this position, it can point to each beamline and acquire images of the IOM optics. These images are analyzed to detect LIDs.

The distance between the IOM and the FODI camera is in the range of 3.7–5.1 m. The size of large aperture optics is $430\mathrm{mm}\times 430\mathrm{mm}$. The resolution of the camera is approximately 124 µm at a working distance of 3.7 m and 136 µm at a working distance of 5.1 m. The CCD image format is $4872\times 3248$ pixels with 12 bits, and the pixel size is $7.4\mathrm{\mu m}\times 7.4\mathrm{\mu m}$.

The online image is required to be able to detect LIDs larger than 150 µm and ensure that the mean relative error (MRE) is less than 15%. Therefore, directly detecting an LID on the surface of the optic with a diameter of less than 100 µm is difficult.

A proper illumination of the optics to be imaged is critical for the detection of small LIDs. Edge illumination is employed here for each optic, which relies on light injected at the proper angle into the edge of the optics that subsequently becomes trapped through total internal reflection (TIR) (Fig. 2). An LID on the optic surface disrupts the TIR and causes light to scatter from the optic at the flaw location. A percentage of this scattered light is collected and imaged by the FODI imaging system, which appears bright against a dark background (named darkfield imaging). Edge illumination lights only the optic to be imaged and provides excellent local signal to noise ratio performance. The 150 µm flaw is readily apparent in the darkfield image.

An edge illumination system was developed specifically for FODI. Two semiconductor laser light source modules were developed and inlayed into the mounts of optics within the IOM (Fig. 3).

The size of initiated LID may be less than 50 µm. The edge illumination is uneven, the damage signal in the dark part may be weak, and the imaging resolution is lower than the LID size. For these tiny LIDs, whether they had grown up is the focus of online detection, but these flaws are too small and difficult to detect or accurately segment (Fig. 4).

Aiming at these problems, we propose a method (Fig. 5), which effectively improves the initial LID detection ability through super-resolution reconstruction (SR) and adaptive segmentation method.

The FODI images are darkfield images with less texture information. Most of the areas in the image are smooth and black, and there are fewer features that can be used to learn for image SR. Therefore, attention mechanisms need to be introduced that can focus the operation of the deep neural network on more important information areas. The choice of a single attention mechanism method can only pay attention to one image level and cannot adapt to the SR requirements, so we propose an image SR improvement method by integrating layers, channel space, planar areas, edges, and corner points^[5,6] in five attention modules to achieve SR (Fig. 5).

The essence of image SR is to learn the regression mapping function from low-resolution (LR) images to high-resolution (HR) images from the features of the LR-HR image pairs, which can achieve an image resolution that exceeds the optical resolution determined by the Rayleigh criterion.

The SR methods have made great strides because of deep learning techniques. The first algorithm to apply convolutional neural networks (CNNs) to image SR in 2014 was SRCNN^[7]. In order to solve the problem of difficult training of deep networks, ResNet^[8] was proposed in 2016. In the same year, subpixel convolutional layers were introduced to overcome the shortcomings of interpolation and learn end-to-end up sampling. In order to avoid overfitting, a deeply recursive convolutional network (DRCN)^[9] constructed by a recurrent neural network (RNN) was proposed. In 2017, in order to take full advantage of the features of the network model at all levels, the SRDenseNet model^[10] introduced SenseNet to single-frame image super-resolution reconstruction, in which each layer of features is fed into all subsequent layers in a dense block, concatenating all layer features instead of directly adding them together like ResNet. In the same year, Zhang et al.^[11] argued that images containing a large amount of low-frequency information hindered the characterization ability of CNNs and proposed the deep residual channel attention network (RCAN). In 2019, Zhang et al.^[12] argued that by down sampling the construction sample images in HR to lose the detail and accuracy that could be obtained from the original data, they trained the depth network by taking real optical zoom images. Using a context bilateral loss, advanced performance is achieved in $4\times$ and $8\times$ computational zooms.

In 2020, progress was made in channel attention, with typical approaches including component divide-and-conquer (CDC)^[5] and holistic attenuation network (HAN)^[6]. The former builds three component-attentive blocks (CABs) associated with flat areas, edges, and corners for three different areas in the image: flat areas, edges, and corners. Each attention block learns the mapping from LR to HR through an intermediate supervision (IS) strategy. The latter adaptively emphasizes hierarchical features by considering the correlation between layers. More information features are captured by learning confidence levels for all locations in each channel. Attention feature maps are generated by capturing joint channels and spatial features.

In 2021, Chen et al. proposed a local hidden function method called local implicit image function (LIIF)^[13] to continuously describe local information of complex images. It is possible to achieve any multi-resolution improvement within 30 times magnification and obtain excellent reconstruction results. LIIF represents the latest level of continuous up sampling and high-quality magnified images. However, the problems of darkfield images are less targeted in this article.

Based on the above analysis, CDC and HAN integrated methods are used in this paper, which make full use of the information of the boundary area of the damage point in the darkfield image for image SR.

We conduct transfer learning and integrate the SR images obtained by CDC (CDC_SR) and HAN (HAN_SR) by adaptive weight and using structural similarity (SSIM) for evaluation to improve the quality of the integrated SR image. The integration method is as follows: $$Y=\mathrm{HAN}\_\mathrm{SR}\times \alpha +\mathrm{CDC}\_\mathrm{SR}\times \beta ,$$(1)where $\alpha$ and $\beta$ are the integration factors.

The final HR image Y can be obtained by minimizing Eq. (2) via the limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) algorithm^[14] as $${\arg \min }_Y(Y|\{\mathrm{HAN}\_\mathrm{SR},\mathrm{CDC}\_\mathrm{SR}\},\mathrm{SSIM}).$$(2)

SSIM is defined as $$\mathrm{SSIM}(I,\widehat{I})=\frac{(2{\mu }_I{\mu }_{\widehat{I}}+C_1)({\sigma }_{I\widehat{I}}+C_2)}{({\mu }_I^2+{\mu }_{\widehat{I}}^2+C_1)({\sigma }_I^2+{\sigma }_{\widehat{I}}^2+C_2)},$$(3)where $I$ is the input original image, $\widehat{I}$ is the output HR image Y, ${\mu }_I$ is the mean value of image $I$; ${\sigma }_I$ is variance of image $I$; ${\sigma }_{I\widehat{I}}$ is covariance of the integrated SR image and the ground truth image; $C_1={(K_{1}L)}^2$, $C_2={(K_{2}L)}^2$, $L$ is the dynamic range of the pixel value of image $I$, $K_1=0.01$, and $K_2=0.03$.

The uneven illumination and the variety of damage types (Fig. 6) make the accurate segmentation of LID a challenging job.

The local area signal-to-noise ratio (LASNR) method^[16] is a meaningful approach that has no theoretical limitations in object size, but some limitations can be imposed by the selection of algorithm parameters for size and signal-to-noise cutoff values.

Other learnable methods can be found in the challenging document image binarization studies. According to how the thresholding values are computed for image binarization, these methods can be divided into histogram-based methods^[17], feature-based methods^[18], and learning-based methods^[19].

In order to avoid relying heavily on the training dataset, we designed an adaptive image segmentation algorithm based on the image SR.

For segmentation of the integrated SR image, for a pixel at position $(r,c)$, a local threshold $T(r,c)$ is calculated within a window of size mask_size × mask_size as follows: $$T(r,c)=u(r,c)\{1+k[\frac{\sigma (r,c)}{R}-1]\},$$(4)where $u(r,c)$ is the local mean value within the window, and $\sigma (r,c)$ denotes the corresponding standard deviation. The parameter $R$ is the assumed maximum value of the standard deviation, and k is a parameter that controls how much the threshold value $T(r,c)$ differs from the mean value. If there is high contrast in the neighborhood of a point $(r,c)$, the standard deviation $\sigma (r,c)$ has a value close to $R$, which yields a threshold value $T(r,c)$ close to the local mean. If the contrast is low, the local threshold is below the local mean value. Every pixel $p(r,c)$ whose gray value is bigger than the calculated local threshold $T(r,c)$ is selected.

The parameter mask_size specifies the size of the neighborhood in which the local threshold is calculated. The smaller the window size, the thinner the segmented flaw. Mask_size must be set to a value that is larger than the LID size in the original image. Mask_size needs to be multiplied by the super-resolution factor $n$ in the SR images.

The parameter k controls how much the threshold value differs from the local mean value. Using a smaller value of k can segment structures with lower contrast to their background. Using larger k values can suppress clutter. Changing k can determine the area of the LID.

The k value varies from large to small (k₁ > k₂ > k₃), and the segmented area of the local seed area also increases. When an LID is super-imposed on a background feature or noise with elevated intensity, reducing the k can include all connected pixels greater than the proper pixels and falsely label the background feature as part of the LID, as shown in Fig. 7.

When the rate of change of k reaches the maximum value, the k is then used as the threshold for the local segmentation.

Figure 8 is the diagram of the FODI online experiment configuration, and Fig. 9 is the actual optical system of IOM and the optical imaging lens.

The IOM contains nine large aperture optics, as shown in Fig. 9, from left to right: vacuum closed window, fundamental frequency cepstral peak performance (CPP), double frequency crystal, mixing crystal 1, mixing crystal 2, wedge lens (convex optic), big-aperture sampling grating (BSG), triple frequency CPP, and vacuum isolation sheet.

Since the IOM contains a wedge-shaped lens, the optimized optical design of the optical imaging lens is associated with IOM’s optics. The main objective group is fixed, and the distance from the IOM remains constant. The focus objective group is a zoom lens group, which changes the focal length by changing the distance between the focus objective group and the main objective. By moving the position of the CCD, the image is maintained clearly, and the magnification of the imaging system is basically constant.

The optical resolution (derived from the modulation transfer function) and pixel resolution (pixel equivalent) are shown in Table 1.

We captured 200 online images and made a dataset. We carried out transfer learning for HAN and CDC SR methods using the dataset by down sampling the online images.

We randomly selected 12 images from the 100 test images to verify the effectiveness of the present method.

The HAN, CDC, and integrated SR images are obtained as shown in Fig. 10.

In this SR experiment, the SSIM value of the integrated SR image was improved by an average of 3%, which proves that the proposed SR integrating method has a lifting effect.

Figure 11 shows an example of one LID in an integrated SR image whose size is less than 100 µm.

Adaptive LID segmentation is performed using the integrated resolution images to detect LIDs. The experimental results are shown in Fig. 12.

In this image segmentation experiment, the number of LIDs was found to be increased 0.6 times and 0.9 times, respectively, in $2\times$ and $4\times$ integrated resolution images, which proves that the proposed adaptive image segmentation algorithm based on the integrating image SR method can improve weak LID detection capabilities. Of course, there may be some false damage points. For further identification methods, please refer to Ref. [4].

The detection rate results of the randomly selected 12 images showed that our method can evenly detect 93.4%, 96.5%, and 97.7% surface LIDs with diameter more than 50 µm, respectively, in the original, $2\times$, and $4\times$ integrated resolution images (as shown in Fig. 13). The corresponding standard deviations are 0.93, 1.09, and 0.98, respectively.

In addition, the boundary locations are subdivided, which helps to improve the accuracy of boundary positioning.

An in-situ LID inspection method was presented based on laser TIR darkfield imaging, image SR integrating, and adaptive segmentation. The method can detect the LIDs with few pixels and uneven lighting conditions. The method can improve the initial damage detection capabilities, and the validity was proved by the experimental results.