Multidimensional Reconstruction of Internal Defects in Additively Manufactured Steel Using Photothermal Super Resolution Combined With Virtual Wave-Based Image Processing

We combine three different approaches to greatly enhance the defect reconstruction ability of active thermographic testing. As experimental approach, laser-based structured illumination is performed in a stepwise manner. As an intermediate signal processing step, the virtual wave concept is used in order to effectively convert the notoriously difficult to solve diffusion-based inverse problem into a somewhat milder wave-based inverse problem. As a final step, a compressed-sensing-based optimization procedure is applied which efficiently solves the inverse problem by making advantage of the joint sparsity of multiple blind measurements. To evaluate our proposed processing technique, we investigate an additively manufactured stainless steel sample with eight internal defects. The concerted super resolution approach is compared to conventional thermographic reconstruction techniques and shows an at least four times better spatial resolution.

SECTION I. Introduction

In RECENT years, active thermographic testing (TT) has gained more and more attention in industry, especially as a contactless, simple, and relatively inexpensive nondestructive testing method. In TT, a heat flow in the specimen is forced by an external heat source. Mostly, light sources such as high-power lasers or flash lamps are employed to make use of the photothermal effect [1], [2]. The resulting heat flow in the material is altered at defect regions, e.g., given by cavities in the material, which is indicated as a temperature change in the measured thermal image sequence by an infrared (IR) camera. The diffusive behavior of heat impedes a precise reconstruction of subsurface defects in a material, especially in metals such as stainless steel. The high thermal conductivity of stainless steel leads to the strong degrading effect that two closely spaced defects could be recognized as one defect [3]. Since stainless steel is a widely used material in the pharmaceutical industry, medical technology, and vehicle construction, particularly due to its corrosion resistance, an improvement of the insufficient conventional thermographic techniques is necessary.

To increase the signal-to-noise ratio (SNR) in the measured thermal images, methods such as lock-in (LIT) or pulsed-phase thermography (PPT) were implemented in the past [4], [5]. These techniques make use of the Fourier transformation (FT), thus investigating amplitude and phase of the signal in the frequency domain. Also thermal pulse-compression has been used for an increased SNR [6]. But the spatial heat blurring remains a major limitation when closely spaced features need to be resolved. Only recently, the disadvantageous diffusion behavior on the spatial resolution of TT could be partly compensated by the introduction of virtual propagating waves using the so-called virtual wave (VW) concept [7]–​[10]. Like FT in LIT or PPT, the VW concept is based on an integral transformation (Fredholm integral equation of the first kind) between the measurement domain, i.e., space × time, and another target domain with the idea to make the original diffusion-based problem more easy to be solved. In FT the target domain is space × frequency, whereas in VW concept it is space × virtual time, which means both are local transformations that can be computed per pixel of the measured thermographic sequence in postprocessing. Applying the VW concept allows for thickness estimations or to determine the defect position in the depth [8].

Apart from that, the spatial resolution can be further enhanced using structured laser illumination as already shown in fields of optics, especially structured illumination (SI) microscopy [11]. By illuminating a surface step by step with a small position shift, a spatial frequency mixing of the illumination pattern and the target pattern is realized. Extracting the high-frequency component from the outcome of spatial frequency mixing is well-known in the literature as optical super resolution (SR) [12]. A transfer of this methodology to thermography led to the development of photothermal SR techniques  [13]–​[15].

It was further shown that compressed-sensing-(CS)-based algorithms benefit from many jointly sparse measurements given by the same sparse defect distribution in each measurement. This approach already demonstrated that it is possible to distinguish closely spaced internal defects in steel [15]. However, thus far these results focused only on one spatial dimension which is enough to distinguish two defects from each other, but not enough to realize quantitative multidimensional defect reconstruction.

In this contribution, we show how the two approaches—photothermal SR and VW concept—can be combined to realize a high-resolution multidimensional defect reconstruction technique. Photothermal SR has so far only focused on the lateral spatial dimension. The combination with the VW concept allows for an additional accurate defect reconstruction in the depth dimension, which is why we call it multidimensional. This means a big step further towards high-resolution photothermal computed tomography.

We first explain the specimen and high-power laser-based experimental setup used for data acquisition. Second, we describe the image processing based on VW concept and CS algorithms which make use of the joint sparsity property in space. Finally, we compare the results with results obtained from conventional thermographic reconstruction techniques based on flash lamp measurements and PPT.

SECTION II. Methodology and Experiments

A. Photothermal SI Experiments

In standard experiments used for TT, a flat heating is preferred such that the entire observation surface can be tested at once. In contrast, the photothermal SR approach presented here, relies on a multitude of similar individual experiments each with a spatially different heating structure enabled by laser SI. We used two different types of high-power near IR laser sources: a fiber-coupled diode laser (max. output power of $P_{{max}} = 530$ W, illumination type: laser line with an area projected onto the sample surface of $A_{\mathrm{laser}}=0.4\times 17 \,\text{mm}^2$) and a vertical-cavity surface-emitting laser (VCSEL) array ($P_{{max}} = 2400$ W, illumination type: 12 laser lines with $A_{\mathrm{laser}}=1\times \text{10}\,\text{mm}^2$, each line is randomly switched on for each measurement, but it is ensured that at least one laser line is radiating). Both laser setups were used to realize step scanning measurements and are shown in Fig. 1(a) and (b). One measurement in step scanning consists of the heating phase (laser pulse length $t_{\mathrm{h}}$ = 500 ms), a cooling phase ($t_{\mathrm{cooling}}$ = 20 s) and a slight position shift of 0.2 mm, repeated for approximately 250 measurements for both configurations (transmission and reflection, see Fig. 1). The scanning strategy with the VCSEL array differs from the single laser (SL): we performed $3 \times 20$ measurements per defect pair. Within the 20 measurements, random illumination patterns of the VCSEL array are chosen, followed by a 0.2 mm position shift. At the end of the 60 measurements there is a large position shift to the next pair of defects (see similar scanning strategy in  [13]). As a result, there is no intentional heating between the defect pairs with the VCSEL array, leading to higher SNR in the VCSEL array data compared to the SL data.

Fig. 1.

(a) Reflection, single laser, (b) Transmission, laser array, (c) Reflection, flash lamps, (d) Transmission, flash lamps, (e) sample geometries. IR camera and the specimen were placed for both configurations—(a) and (b) on a linear table so that both components are moved simultaneously. (e) Specimen with eight cavities and different distances is shown. The arrows over the specimen in (a) and (b) represent the direction of the motion. In all experiments the IR camera is observing the side of the specimen with a 2 mm coverage over the defects.

For the SI experiments we have used a midwave InSb-based IR camera (Infratec IR9300, full frame rate: $f_{\mathrm{cam}}$ = 100 Hz, image size: $A_{\mathrm{cam}}=1280 \times 1024$ pixel, spectral range: $\lambda _{\mathrm{cam}}=3$–$5\,\mathrm{\mu m}$) triggered by a photodiode that recognizes when the laser is turned on. We measured around 1000 frames from the start of the pulse for each measurement. In all experiments, i.e., reflection and transmission configuration, the IR camera is facing the thicker side ($2 \,$mm distance to the defects) of the specimen.

B. Flat Illumination Reference Experiments

To compare these SI step scan measurements with conventional thermographic setups, we also acquired data using two flash lamps (electrical input energy of 12 kJ, the pulse duration of the flash excitation was about $t_{\mathrm{h}}\,\approx \,2$ ms) for thermal excitation. These flash lamps enabled a flat illumination scheme and such a flat heating of the entire surface observed. The measurement setups for reflection and transmission mode are shown schematically in Fig. 1(c) and (d). The temperature was measured with a midwave IR camera (FLIR X8400sc, $f_{\mathrm{cam}}$ = 100 Hz, $A_{\mathrm{cam}}=1280 \times 1024$ pixel, $\lambda _{\mathrm{cam}}=1.5$–$5.1\,\mathrm{\mu m}$). The spatial resolution was approx. $57\,\mathrm{\mu m}$ per pixel. The measuring time was about 10 seconds and was estimated on the basis of the diffusion time $t_{\mathrm{d}}=L^2/\alpha$  which is related to the thermal diffusivity $\alpha$ and the sample thickness $L$.

As reference signal processing method we use PPT, which calculates the phase difference from the temporal FT of the cooling transient after pulsed heating of each pixel to a reference pixel located in a defect-free region. Since we are in frequency domain, we can calculate the frequencies of interest by making use of $f = 1/{t_{\mathrm{d}}} = \alpha /L^2$. Assuming e.g. $L = 4.5$ mm results in $f \approx 0.2$ Hz. For this reason we only show the frequencies in this region in the results section.

SECTION III. Image Processing Methods and Mathematical Description

The whole image processing procedure is depicted in Fig. 2 and was applied to obtain the final reconstruction results. All the relevant steps will be explained in the following.

Fig. 2.

Flow chart depicting the data analysis procedure discussed in this work. The direction of data flow is from left to right. The inspection of intermediate results is marked by empty circles.

A. Mathematical Model for Acquired Data

To create a simple and correct mathematical framework based on the measured temperature from the IR camera, we introduce $T_{ry} \in \mathbb {R}^{N_r\times N_y \times N_t}$ describing discrete absolute temperature values and $\Delta T_{ry}$ describing relative temperature differences. We use the following relation for each measurement $m \in \lbrace 1,\ldots, N_{\mathrm{meas}}\rbrace$: $$\begin{equation*} T_{ry}^m = T_{ry,0}^m + \Delta T_{ry}^m \, \tag{1} \end{equation*}$$ whereby $T_{ry,0}^m$ designates the first measured frame in the 3-D array $T_{ry}^m$, more precisely $T_{ry,0}^m = T_{ry}^m[:,:,1]$. Each measurement $m$ differs due to the differently chosen illumination in the photothermal SI experiment. The $z$-dimension (representing the depth) is not considered, because the IR camera only observes the surface of the specimen. The discretized values are described by squared brackets, whereby $i$, $j$, $k$ stand for indices with $r[i] = i \cdot \Delta r$, $y[j] = j \cdot \Delta y$ and $t[k] = k \cdot \Delta t$ representing two spatial and one time dimension. The conversion from the continuous function works as follows: $T_{ry}^m(r[i],y[j],t[k]) = T_{ry}^m(i \cdot \Delta r, j \cdot \Delta y, k \cdot \Delta t):= T_{ry}^m[i,j,k]$ with $i \in \lbrace 1,\ldots, N_r\rbrace$, $j \in \lbrace 1,\ldots, N_y\rbrace$, and $k \in \lbrace 1,\ldots, N_t\rbrace$. $\Delta r$, $\Delta y$, or $\Delta t$ stand for the resolution in each dimension as described in the experimental setup. Thus, e.g. $T_{ry}^m[2,j,k]$ represents the second pixel temperature value in the horizontal dimension and has a geometrical distance of $\Delta r$ to the pixel with the temperature value $T_{ry}^m[1,j,k]$.

We simplified the model by eliminating the spatial dimension $y$ since we are interested in resolving closely spaced linear defects arranged along $y$ (c.f. Fig. 1). Consequently, we work with $\Delta T^m \in \mathbb {R}^{N_r \times N_t}$ instead of $\Delta T_{ry}^m$. Further, we can describe $\Delta T^m$ using the Green's function approach as discretizing a solution of the heat diffusion equation for a line source and a plate with finite thickness ($\Phi$) [14]–​[16] $$\begin{equation*} \Delta T^m = \Phi *_r x^m. \tag{2} \end{equation*}$$ The operator $*_r$ stands for a convolution in the space domain, $x^m = I_r^m \circ a$ (given in W/m) designates a pointwise product of the illumination $I_r^m \in \mathbb {R}^{N_r}$, and the absorption coefficients in space $a \in \mathbb {R}^{N_r}$. As discussed in detail in  [14], the consideration of the illumination pulse length in $\Phi$ and the consideration of both—the illumination positions and the illumination line width—in $x^m$ is most useful for the reconstruction of defects in blind photothermal SR applications. Thus, $x^m$ is not time-dependent. The initial goal is to obtain $x^m \in \mathbb {R}^{N_r}$ from the solution of the inverse problem formulated in (2) and from that the distribution of $a$, which describes the defect positions. The reason for not computing directly the distribution of $a$ is that we use blind illumination, which means that we do not specify the individual illumination positions $I_r^m$ but rather treat them as part of the variable of interest $x^m$. Therefore, the inaccurate knowledge about $I_r^m$ leads us to first recover all $x^m$ and then to compute $a$. Although this approach increases the complexity of the problem it also makes the solution more robust against small deviations in position, as e.g., the positional noise of a robot holding the IR camera can lead to high degradation in the final result.

B. VW Transformation of Photothermal SI Data

In this work, we propose an image processing technique based on the VW concept that enables precise reconstruction in depth of lateral photothermal SI data. This ansatz transforms our main model equation stated in (2) into the new one $$\begin{equation*} \Delta T_{\mathrm{virt}}^m = \Phi _{\mathrm{virt}} \ast _r x^m. \tag{3} \end{equation*}$$ The desired solution $x^m$ does not change by the application of the VW concept since this is a transformation in time domain and $x^m$ does not depend on time. In this section, we explain how to calculate $\Delta T_{\mathrm{virt}}^m$ from our measured surface temperature data $\Delta T^m$. An analytical derivation of $\Phi _{\mathrm{virt}}$ follows in the next section.

For a temporal heating with a Dirac delta distribution, i.e., infinitely short heating period, the transformation is given in discrete form by [7] $$\begin{align*} \Delta T^m_{\delta } = \Delta T_{\mathrm{virt}}^m K\, \tag{4} \end{align*}$$ which is itself a severely ill-posed inverse problem. The dimensions of the matrices are $\Delta T^m_{\delta } \in \mathbb {R}^{N_r\times N_{t}}$, $K \in \mathbb {R}^{N_{t^{\prime }}\times N_{t}}$, and $T_{\mathrm{virt}}^m \in \mathbb {R}^{N_{r} \times N_{t^{\prime }}}$. The entries of the discrete kernel $K$ depend on the different discrete time scales $\Delta t$ and $\Delta t^{\prime }$ [17] $$\begin{align*} K[l,k] = \frac{\tilde{c}}{\sqrt{\pi \Delta \text{Fo}\,k}} \exp \left(-\frac{\tilde{c}^2(l-1)^2}{4 \Delta \text{Fo}\, k}\right). \tag{5} \end{align*}$$ Herein $\Delta \text{Fo}= \alpha \Delta t/\Delta r^2$ and $\tilde{c} = c \Delta t^{\prime }/\Delta r$ are the discrete Fourier number and the dimensionless virtual speed of sound, respectively. They contain the thermal diffusivity $\alpha$ and the virtual speed of sound $c$ which are the characteristic parameters for thermal wave and “acoustic” VW propagation, respectively. Similar to FT, the kernel of the VW transformation does not depend on position, therefore each data pixel is transformed separately from temperature in time domain into a VW in virtual time domain.

Because of the finite temporal heating time $t_h$, the matrix $\Delta T^m_{\delta }$ has to be convoluted by the matrix $I_t$ considering the corresponding temporal heating [18], which refers to the laser pulse length $$\begin{align*} \Delta T^m &= \Delta T^m_{\delta } *_t I_t = \Delta T_{\mathrm{virt}}^m K *_t I_t. \tag{6} \end{align*}$$

Prior works have shown that incorporating positivity and sparsity as prior information significantly increases the quality of the regularized solution. The multidimensional VW shows a bimodal characteristic equal to the multidimensional photoacoustic wave [19] that results in negative data points and positivity as prior information is not directly applicable. Therefore, we transform these negative data points onto a positive data set by introducing the circular means $\Delta M^m \in \mathbb {R}^{N_{r} \times N_{t^{\prime }}}$ of the VW field. This is possible because of the cylindrical symmetry along $y$ (c.f. Fig. 1) giving rise to 2-D VW data. The relationship between VW signal and spherical means is given by the Abel transform $A \in \mathbb {R}^{N_{t^{\prime }}\times N_{t}}$ [20] $$\begin{align*} \Delta M^m & = \Delta T_{\mathrm{virt}}^m A^{-1} \, \tag{7}\\ A^{-1}[l,k] & = \frac{2 \tilde{c}}{\pi } \frac{1}{\mathrm{Re}(\sqrt{k^2-\tilde{c}^2l^2})}. \tag{8} \end{align*}$$

The original inverse problem for $\Delta T_{\mathrm{virt}}^m$ in (6) has turned into a new one with a much smaller solution space and is solved by means of the alternating direction method of multipliers (ADMM), where we minimize the following cost function: $$\begin{align*} \min f(\Delta M^m[i,:]) + g(\boldsymbol{\gamma }) \quad \text{subject to} \quad \Delta M^m[i,:]-\boldsymbol{\gamma }=\mathbf {0} \tag{9} \end{align*}$$ with $f(\Delta M^m[i,:]) = \Vert \Delta M^m[i,:] A K*_t I_t -\Delta T^m[i,:]\Vert _2^2$ and $g(\boldsymbol{\gamma }) = \lambda \Vert \boldsymbol{\gamma }\Vert _1$ [8], [21], [22]. Herein, $\lambda$ is a regularization parameter that can be determined, for instance, by the L-curve method [23]. To enforce positivity we apply soft thresholding on the positive entries and set the nonpositive entries to zero. The threshold value is defined as the ratio of $\lambda$ and the penalty parameter $\mu$ > 0 (e.g., see  [21]). The $\ell _1$-norm promotes sparsity to the minimizer. To obtain the VW field $\Delta T_{\mathrm{virt}}^m$ based on the spherical means $\Delta M^m$ we apply again the Abel transform $A$ according to (7) [8]. For the results illustrated in Section IV the used number of iteration steps was $N_{\mathrm{iter}}^{\mathrm{ADMM}} =$ 50.

C. Analytical Solution for the VW Equation

To determine $\Phi _{\rm {virt}}$ we made an approximation due to the lack of exact analytical solutions for our investigated case: We calculate $\Phi _{\rm {virt}}$ based on the analytical solution for a finite thin plate where the thickness equals to the distance between the sample surface and the defect position (see analogy to the determination of $\Phi$ in  [15]).

First, we calculate $\varphi _{\rm {virt}}(r,z,t^{\prime })$ for the homogeneous VW equation $$\begin{align*} \frac{\partial ^2\varphi _{\rm {virt}}(r,z,t^{\prime })}{\partial r^2}+\frac{\partial ^2\varphi _{\rm {virt}}(r,z,t^{\prime })}{\partial z^2}- \frac{1}{c^2}\frac{\partial ^2\varphi _{\rm {virt}}(r,z,t^{\prime })}{\partial t^{\prime 2}}=0 \tag{10} \end{align*}$$ for the initial source distribution $\varphi _0(r,z)$ and adiabatic boundary conditions using the cosine transform, which results in $$\begin{align*} \theta _{\rm {virt}}(\xi _r,\xi _z,t^{\prime }) = \theta _0(\xi _r,\xi _z) \cos \left(\sqrt{\xi _r^2+\xi _z^2} c t^{\prime }\right). \tag{11} \end{align*}$$ Herein, $\theta _{\rm {virt}}$ and $\theta _0$ are the cosine transforms of $\varphi _{\rm {virt}}$ and $\varphi _0$, respectively. $\xi _r$ and $\xi _z$ denote the spatial frequencies and $c$ is the virtual speed of sound. To compute $\varphi _{\rm {virt}}$ we carried out the inverse cosine transform of $\theta _{\rm {virt}}$. To obtain $\Phi _{\rm {virt}}\in \mathbb {R}^{N_r \times N_{t^{\prime }}}$ for the reflection and transmission mode, we evaluated $\varphi _{\rm {virt}}$ at $z=0$ (observation plane) for different depth positions $\zeta =ct^{\prime }$ of the pointlike source $$\begin{align*} \Phi _0 = {\begin{cases}1 \,\text{K}, \quad r = L_r/2\; \wedge \; \zeta = \zeta _0\\ 0, \quad \text{elsewhere} \end{cases}} \tag{12} \end{align*}$$ wherein $L_r$ is the width of the sample, $\zeta _0 = ct^{\prime }_0$ with $t^{\prime }_0$ standing for the time at which a change or increase of the virtual temperature at the observation plane becomes visible after excitation.

In the following, $\zeta$ is designated as the virtual depth as it is derived from the analyzed virtual time step $t^{\prime }$ and therefore denotes the path length of the VW at a certain virtual time step.

D. Data Size Reduction for 1-D Reconstruction

Due to the huge amount of data given by the SI-based laser step scanning and the poor computational performance of the algorithm we use in our next processing step, we decided to extract one vector $\in \mathbb {R}^{N_r}$ from each measurement sequence for 1-D reconstruction (max. contrast index in Fig. 2). We extracted the 1-D vector with the highest SNR, e.g., the frame at $\zeta _{\mathrm{{max}}}=ct^{\prime }_{\mathrm{{max}}}=6$ mm in the virtual B-scan for transmission configuration of our VW transformed data [see defect echo in Fig. 5(b), (d)]. Consequently, this data reduction also has to be applied to the $t^{\prime }$-dependent $\Phi _{\mathrm{virt}}$ at the same position $\zeta =\zeta _{\mathrm{{max}}}$ (see black dashed line in Fig. 3). Therefore, $\Delta T^m_{\mathrm{virt}}$ changes to $\Delta T^m_{\mathrm{virt}}(r,t^{\prime }=\zeta _{\mathrm{{max}}}/c) =\Delta T^m_{\mathrm{virt,\,rec}} \in \mathbb {R}^{N_r}$ and $\Phi _{\mathrm{virt}}$ changes to $\Phi _{\mathrm{virt}}(r,t^{\prime }=\zeta _{\mathrm{{max}}}/c)=\Phi _{\mathrm{virt,\,rec}} \in \mathbb {R}^{N_r}$. Hence, the equation for the mathematical model changes from (3) to $$\begin{equation*} \Delta T_{\mathrm{virt,\,rec}}^m = \Phi _{\mathrm{virt,\,rec}} *_r x^m. \tag{13} \end{equation*}$$ In  [13] it was shown that one can perform ultrasound reconstruction techniques such as the frequency domain synthetic aperture focusing technique (F-SAFT) benefiting from the whole virtual time domain. Since the application of F-SAFT did not contribute to significant improvements within these studies, we refer to the maximum SNR method only which makes the whole processing procedure simple and easy to use.

Fig. 3.

(a) $\Phi _{\mathrm{virt}}$ for a source depth of $\zeta =0$, (b) same for $\zeta = 6$ mm. $\Phi _{{virt}}$ was calculated according to Section III-C. The dashed black line indicates the vector $\Phi _{\mathrm{virt,rec}}$. The factor $1/2$ is added in the y-label in (a) due to the double propagation path in reflection configuration.

Fig. 4.

Comparison of 2-D flash results. (a) Flash Tx Raw $\Delta{\mathrm{T}} [\rm K]$, (b) Flash Tx VW $\Delta {\rm T}_{\rm virt} [\rm {K}]$, (c) Flash Tx PPT $\Delta_\phi$ [rad] (d) Flash Rx Raw $\Delta{\mathrm{T}} [\rm {K}]$, (e) Flash Rx VW $\Delta {\rm T}_{\rm virt}$ [K], (f) Flash Rx VW $\Delta_\phi$ [rad]. (a) and (d): Raw temperature difference $\Delta T$ for reflection (Rx) and transmission (Tx) configuration, as indicated. (b) and (e): VW applied to (a) and (d) using the regularization parameters in Table I. (c) and (f): PPT result of (a) and (d) by calculating the phase difference $\Delta \phi _{\mathrm{Tx}} = \phi _{\mathrm{Tx}} - \phi _{\mathrm{Tx,\,ref}}$, whereby $\phi _{\mathrm{Tx,\,ref}}$ refers to a pixel in a defect-free region.

Fig. 5.

Comparison of 2-D raw SI data and 2-D SI results after VW reconstruction. (a) SI, Tx, VCSEL, (b) SI+VW, Tx, VCSEL, (c) SI, Tx, VCSEL, (d) SI+ VW, Tx, SL, (e) SI, Rx, SL, (f) SI+VW,Rx, VCSEL, (g) SI, Rx, SL, (h) SI+VW,Rx, SL. (a), (c), (e), and (g): Mean over raw data $\sum _{m=1}^{N_{\mathrm{meas}}} \Delta T^m / N_{\mathrm{meas}}$ for Tx and Rx for the VCSEL and single laser (SL) experiments, as indicated. (b), (d), (f), and (h): $\Delta T_{{virt, rec, 2D}}$, i.e., VW transform applied to the raw data in (a), (c), (e), and (g) using the regularization parameters in Table I. $\zeta -L$ is shown in (b), (d) because the first signal, related to a single one-way pass of the virtual wave from backside to front side, in transmission comes at $\zeta = L$ and signals related to defects appear later or at a higher virtual depth $\zeta > L$, respectively. Due to the doubled distance travelled by the virtual wave in reflection, $\zeta /2$ is displayed in (f) and (h). The dashed red lines indicate the defects (i.e., defect echo).

E. Iterative Optimization in the Joint Sparsity Domain

We treat the final inverse problem given in (13) by application of the CS-based iterative joint sparsity (IJOSP) algorithm. The IJOSP algorithm promotes jointly sparse solutions, especially for blind illumination, $\hat{x} \approx x$ (assuming $x$ is the real defect pattern/ ground truth) containing the defect pattern in space by considering the following minimization problem [3]: $$\begin{align*} \hat{x} = & \arg \min _{\tilde{x}} \sum _m\Vert \Phi _{\mathrm{virt,\,rec}} \ast _r \tilde{x}^m\\ &- \Delta T_{\mathrm{virt,\,rec}}^m\Vert _2^2 + \lambda _1 \Vert \tilde{x}\Vert _{2,1} + \frac{\lambda _2}{2} \Vert \tilde{x}\Vert _2^2 \tag{14} \end{align*}$$ $\lambda _1$ and $\lambda _2$ are controlling the impact of the regularizers, more precisely, the impact of block sparsity via $\Vert \tilde{x}\Vert _{2,1} = \sum _{i=1}^{N_r} \sqrt{\sum _{m=1}^{N_{\mathrm{meas}}} |\tilde{x}^m[i]|^2}$ and Tikhonov regularization $\Vert \tilde{x}\Vert _2^2$. Thus, the block sparsity has a positive influence within the optimization algorithm if the local optimization variable $\tilde{x}$ exhibits sparsity in the spatial dimension $r$ with a great similarity over all measurements $m$. The main reason for using the combined $\ell _{2,1}$-norm instead of the $\ell _1$-norm is due to the use of blind illumination. As mentioned above, we do not use the spatial illumination information in $x^m$. Furthermore, such an approach could serve for a greater robustness against shortcomings in the exact knowledge of $\Phi _{\mathrm{virt,\,rec}}$. The additional consideration of the Tikhonov regularization can lead to more realistic defect reconstruction by means of less sparse solutions and more degrees of freedom in thresholding within the optimization routines (see  [14]).

To find a solution $\hat{x}$ for the underlying underdetermined system, various approaches can be applied to solve (14). In this work, we have used iterative optimization algorithms such as Block-FISTA and Block-Elastic-Net [14], [24], [25]. In Block-FISTA, we do not make use of the Tikhonov regularization so that $\lambda _2 = 0$. Both optimization methods use an updating step size related factor $L_c$ within the gradient descent implementation which designates the Lipschitz constant of the gradient of the least squares term. All shown results were achieved using 500 iterations leading empirically to convergence.

F. Visualization of Multidimensional Reconstruction Results

The results in the next section show 2-D results based on $\Delta T_{{virt, rec, 2D}} = \sum _m \Delta T_{\mathrm{virt}}^m \in \mathbb {R}^{N_r\times N_{t^{\prime }}}$ and 1-D results based on the normalized $\hat{x}_{{rec}} = \sum _m \hat{x}^m / \max (\sum _m \hat{x}^m) \in \mathbb {R}^{N_r}$ (cf. Fig. 2). The reason for calculating the sum over the measurements $m$ is due to the fact that we performed measurements that are overlapping (SR). Due to the overlap and the laser scan over the whole surface of the specimen, the normalized sum of the illumination patterns over all measurements is equal to a normalized homogeneous illumination over the whole surface (i.e., $1 =$ illumination, $0 =$ no illumination) so that we can write: $\sum _{m} I_r^m / \max (\sum _{m} I_r^m) \approx I_{r,\,\text{homogeneous}}= J_{1,N_r}$, whereby $J_{1,N_r}$ designates an all-one vector with $1 \times N_r$ elements. Therefore, using $\hat{x}_{{rec}}=\sum _m \hat{x}^m / \max (\sum _m \hat{x}^m) \approx \sum _m x^m / \max (\sum _m x^m) = \sum _m a \circ I_r^m / \max (\sum _m a \circ I_r^m) \approx a / \max (a)$, pretty good visualizes the absorption pattern $a$ in our photothermal SR experiments.

SECTION IV. Results and Discussion

A. Reconstruction From Conventional Flash Thermography

At first we show the results in Fig. 4 obtained by using a flash lamp (i.e., flat illumination) in transmission and reflection configuration. We decided to compare the measured temperature difference (raw) with the PPT as an advanced conventional thermographic technique in comparison with our self-developed VW concept.

It is clearly visible that we obtain much better results in transmission than in reflection configuration for both processing techniques. The results obtained in reflection using the flash lamp are useless independent of which image processing method is used. In contrast, the transmission results show very good indications for the first two defect pairs with the largest distances to each other—aspect ratio (AR) of 2:1 and 1:1 (see Fig. 1). A moderately good indication for the third defect pair (AR = 1:2) can also be observed in Fig. 4(a)–​(c). The VW image in Fig. 4(b) exhibits the most clear result providing a high SNR. Another conventional approach besides flash thermography can be realized by the application of lock-in thermography measurements. However,  [26] showcases limitations in spatial resolution already at AR = 1:1, also with a sample made of stainless steel, which did not encourage us to make use of lock-in thermography.

Further, as explained in Section III-B we are able to illustrate the image in depth dimension using the VW concept. The result is a virtual B-scan exhibiting reflections like in ultrasonic testing that partly compensates the heat blurring seen in the raw data. Regarding lateral resolution, the PPT result in Fig. 4(c) exhibits less heat blurring than the raw data shown in Fig. 4(a).

B. 2-D Reconstruction From SI Laser Thermography

The 2-D results using SI (i.e., $\sum _{m=1}^{N_{\mathrm{meas}}} \Delta T^m$ and $\Delta T_{{virt, rec, 2-D}}$) described in this section were obtained as shown in Fig. 2 graphically. We first show the 2-D results under consideration of the lateral and depth resolution (i.e., in $\mathbb {R}^{N_r\times N_{t}}$ and $\mathbb {R}^{N_r\times N_{t^{\prime }}}$, respectively) in Fig. 5. Comparing these results with the results from Fig. 4 where we used a flash lamp (flat illumination), we achieve much more accurate results and can even separate the defects in the fourth defect pair (AR = 1:4). Note that the SI VCSEL array data provides higher SNR at the defect area than the SL since we have used different scanning strategies (cf. Section II-A).

Since VW allows for a virtual B-scan, we cannot only reconstruct the defect along the lateral dimension but also in the depth. As discussed earlier in Section III-D, we identify the signals in Fig. 5(b), (d), (f), (h) showing the highest SNRs with the defect and backwall echos. The backwall echos are best seen in transmission from the depth $\zeta \approx 4.5\,$mm, whereas the defect related echos are seen in reflection at $\zeta \approx 4\,$mm.

$\zeta$ is directly related to the traveled path length of the VW and therefore represents a kind of virtual depth. In reflection configuration, the VW is released from the front side of the sample and reflected back from the defect onto the same side which also serves as the observation plane using the IR camera. For defect depth estimation we therefore have to either halve the virtual speed of sound $c$ (see e.g.,  [9]) or the virtual depth $\zeta$ to compensate for the doubled propagation path.

Hence, in reflection configuration [c.f. Fig. 5(f), (h)] the defects can be detected at a depth of around $\zeta /2 \approx 4a = 2\,$mm (c.f. Fig. 1).

In transmission configuration, the traveled path length to be considered is more complex. Now the VW is released from the backside of the sample and may experience reflections at a given defect and the backside of the sample before propagating to the front side for observation. Thus, compared to a VW that propagates through a defect-free region, the VW that interacts with a defect is detected with a certain delay. This delay depends on defect width, defect depth and can further be influenced by the fact that the VW source does not represent an ideal point source. To highlight these defect depth related delays we plot the transmission results as $\zeta - L$ [c.f. Fig. 5(b), (d)] and observe the defect echos at $\zeta - L \approx 1\dots 2 \,$mm, which is indeed compatible with assuming a reflection of the VW from the backside of the defect (distance of $a=0.5 \,$mm from illuminated backside) before propagating through the total thickness ($L=4.5 \,$mm) of the sample. As a consequence, we used the values of $\zeta _{\mathrm{max}} = 6 \,$mm (transmission) and 2 mm (reflection) for calculating $\Phi _{\mathrm{virt,\,rec}}$ in the reconstruction of $\hat{x}_{{rec}}$.

C. 1-D Reconstruction From Flash and SR Laser Thermography

The 1-D reconstruction results for flash in Fig. 6(a) and (b) confirm our previous statements. The conventional thermographic method using flash lamps and PPT is clearly improved by applying the VW transform in postprocessing. In transmission, we can see indications for all four defect pairs (AR = 1:4). For reflection, however, no progress is seen using any of the reconstruction, probably because of insufficient heating by the flash lamps.

Fig. 6.

Comparison of normalized 1-D results from all techniques with the true defect shapes (grey shaded area). (a) Flash results, Tx, (b) Flash results, Rx, (c) VCSEL SR results, Tx, (d) VCSEL SR results, Rx, (e) Single laser SR results, Tx, (f) Single laser SR results, Rx. Left column (a), (c), and (e) is for transmission and right column (b), (d), (f) for reflection configuration, measured with flash lamps (a), (b), VCSEL array (c), (d) and single laser (e), (f), respectively. The different plots indicate results for raw data [absolute values from Fig. 4 over time intervals $0.7\dots 1\,$s, 20 s for (a), (b)], PPT data (from Fig. 4 for $\Delta \phi [:,f_0 = \text{0.29}\,\text{Hz}]$), VW data [from Fig. 4 at $z=7$ mm, $0\dots 2.7$ mm for (a), (b)], SI data (abs. temp. difference from Fig. 5 averaged over $t\,=0.5\dots 1 \,$s, $0\dots 1 \,$s, $0.5\dots 1 \,$s, $0.01\dots 5 \,$s for (c), (d), (e), and (f)), SI+VW data [from Fig. 5 for depths of $5.8\dots 5.9 \,$mm, 2 mm, $5.1\dots 5.3 \,$mm, $2.32\dots 2.35 \,$mm, for (c), (d), (e), (f)] and SR = SI+VW+IJOSP data [$\hat{x}$, see flow chart in Fig. 2 for (c), (d), (e), (f)]. The used regularization parameters are listed in Table I.

The 1-D results displayed in Fig. 6(c)–​(f) show even more improved results. We observe a much better defect reconstruction after applying the VW transform to the raw SI data. The most sparse and best reconstructions of the defect pairs are obtained using a combination of the experimental SI procedure together with the VW and CS based IJOSP in postprocessing, which forms our final SR method, because it additionally benefits from the joint sparsity property given by multiple measurements. Comparing the two laser types, we do not see a clear winner. The VCSEL array seems to perform better in reflection, whereas the SL performs better in transmission configuration. A slight advantage of the SL can be seen in terms of spatial resolution, since it has a narrower spot shape (i.e., 0.4 mm versus 1.0 mm). The final reconstruction quality, however, depends on a multitude of parameters, for instance on the number of measurements [13].

Table II shows the improvement of the reconstruction accuracy by using the VWC and SR for all illustrated curves in Fig. 6. The calculated reconstruction accuracy is a quantitative metric for the reconstruction of width and amplitude of the defects. Moreover, it considers the success of separating two defects within a defect pair (see formulas to calculate reconstruction accuracy in  [15]).

TABLE I Chosen Regularization Parameters to Obtain the Results from Different Configurations Shown in Figs. 4–6

TABLE II Overall Reconstruction Accuracies Are Calculated as Shown in [15]. This Metric Is Used to Evaluate the Separability of Defects. Hence, the Reconstruction Accuracies Are in [0,1], Where 1 Stands for a Perfect Reconstruction and 0 Denotes That No Indication Is Given for Two Defects Within One Investigated Defect Pair

Comparing these results with the studies in  [15] where the maximum thermogram method (MT) was used, we see a huge improvement using the VW concept in transmission configuration [see almost perfect reconstruction of defects in Fig. 6(e)]. Unfortunately, we do not observe such an improvement in reflection. We suspect that the reason is that the VW concept is implemented by a local transformation which increases the SNR pixelwise such that in data exhibiting similarly high SNRs for each pixel, nondefect areas can be interpreted as defect areas by the VW algorithm. In contrast, MT extracts one thermogram without changing the spatial shape of the thermal image. These studies therefore encourage the application of the VW concept in reflection with high-power heating sources and many measurements.

SECTION V. Conclusion

The combination of photothermal SI measurements and VW plus IJOSP image reconstruction represented a novel SR reconstruction technique for subsurface defects in metals. Especially in reflection configuration, we showed stunning results outperforming conventional flash thermography with PPT by far. Hidden defects in additively manufactured stainless steel with a defect distance to depth ratio of 1:4 were almost perfectly reconstructed. With this new proposed method, we had shown how to beat the deteriorated spatial resolvability given by heat diffusion. It is highly recommended to use this method in fields of nondestructive testing due to the given defect sparsity. In our multistep approach, we had identified a number of critical points for its success: First, photothermal SI experiments demanded for structured heating using high-power laser sources and several measurements. Second, the used computational methods required a high degree of accuracy and physical understanding and were crucial for obtaining high reconstruction qualities in the final result. Further, regularization parameters had to be chosen which was a main bottleneck in this procedure. As an outlook, we work on deep-learning-based regularization, to calculate layer by layer new parameters for faster and perhaps also more accurate reconstruction results.