Deep D-Bar: Real-Time Electrical Impedance Tomography Imaging With Deep Neural Networks

The mathematical problem for electrical impedance tomography (EIT) is a highly nonlinear ill-posed inverse problem requiring carefully designed reconstruction procedures to ensure reliable image generation. D-bar methods are based on a rigorous mathematical analysis and provide robust direct reconstructions by using a low-pass filtering of the associated nonlinear Fourier data. Similarly to low-pass filtering of linear Fourier data, only using low frequencies in the image recovery process results in blurred images lacking sharp features, such as clear organ boundaries. Convolutional neural networks provide a powerful framework for post-processing such convolved direct reconstructions. In this paper, we demonstrate that these CNN techniques lead to sharp and reliable reconstructions even for the highly nonlinear inverse problem of EIT. The network is trained on data sets of simulated examples and then applied to experimental data without the need to perform an additional transfer training. Results for absolute EIT images are presented using experimental EIT data from the ACT4 and KIT4 EIT systems.

SECTION I. Introduction

Electrical Impedance Tomography (EIT) images traditionally display the tissue-dependent conductivity distribution of a patient in the plane of the attached measurement electrodes allowing, e.g., visualization of heart and lung function as well as injuries [1]–​[6]. The resulting images are of high-contrast and data acquisition is done by harmless electrical measurements without the need for contrast agents or ionizing radiation. However, the image recovery process of forming the EIT image from the current/voltage measurement data is a severely ill-posed nonlinear inverse problem, and thus requires a noise-robust regularization strategy for stability. The ‘D-bar method’, the only proven regularization strategy for the full nonlinear problem [7], provides real-time noise-robust image recovery by using a low-pass filter of the associated nonlinear Fourier data. Unfortunately, this results in images that suffer a loss of sharp features often important in medical imaging applications. In this work, we propose combining D-bar with Deep Learning, specifically with a Convolutional Neural Network, to ‘learn’ and undo the image blurring resulting in real-time sharp EIT images.

EIT reconstructions are typically computed with iterative algorithms that are based on minimizing a penalty functional, such as [8] and [9]. These methods perform very well in reconstruction quality due to a flexibility of incorporating prior knowledge, but require careful modeling of the boundary shape in the repeated simulation of the forward problem. Possibilities to overcome the boundary sensitivity are proposed in [10] and [11], but tend to be computationally demanding. On the other hand, direct (non-iterative) reconstruction algorithms do not need the repeated simulation of the forward operator. One such method is known as the D-bar algorithm which is based on a nonlinear Fourier transformation of the measured surface current/voltage data. The method employs a low-pass filtering of this transformed data as a regularization strategy to stabilize the image reconstruction process against noise in the measured data. Consequently, this filtering results in reconstructed images that suffer from a significant loss of sharpness. It has been shown that the direct D-bar method is robust to incorrect or incomplete knowledge of electrode locations as well as errors in boundary shape, see for instance [12] and the discussion in Section II-B. Iterative methods on the other hand are either very sensitive to the correct forward model or are based on sophisticated modelling to cope with uncertainties in the model, such as unknown electrode locations, boundary shape, or contact impedances [10], [11], [13].

Recent advances in the larger field of image reconstruction have demonstrated the power of Deep Learning and Neural Networks for improving low quality or corrupted images. In particular, combining fast direct reconstruction procedures with deep neural networks can provide high quality images with low latency, leading to prospective real-time imaging in many applications. Convolutional Neural Networks (CNN) are especially suitable for post-processing initial reconstructions that come from algorithms based on, or related to, Fourier transforms, as suggested in [14]. Such initial reconstructions typically suffer from a loss of spatial resolution, due to some sort of low-pass filtering, as well as additional undersampling artefacts. Training a CNN to remove these artefacts to improve the information content of the reconstructed image has been studied for several linear inverse problems in medical imaging, including CT [14], [15], MRI [16], and PAT [17], [18]. Although the EIT problem is nonlinear in nature, the low-pass filtered images from the low-passed D-bar method naturally fit into this setting.

In this study we formulate a real-time capable reconstruction algorithm that produces high quality sharp absolute EIT images by combining the D-bar algorithm with subsequent processing by a CNN. For this task we utilize an established CNN architecture, known as U-net, adjusted to cope with the typical image structures of D-bar EIT reconstructions. We train the network on simulated training data and directly apply the trained network to experimental data with no training on experimental data itself. This successful transition to experimental data underlines the robustness of the D-bar algorithm and is especially important as the need for good training data is often the bottleneck for the success of such network-based approaches for other imaging modalities, [14], [18], [19].

This paper is organized as follows. Section II presents a brief review of the mathematical problem of EIT and the D-bar solution method. The deep learning CNN for D-bar, coined ‘Deep D-bar’ is introduced in Section III. The experimental setup as well as simulation of training data are described in Section IV and results presented in Section V. A discussion of the results is given in Section VI and conclusions drawn in Section VII. The reader is encouraged to view the manuscript on a computer screen as details in the image contrast may be masked in printed versions.

SECTION II. Electrical Impedance Tomography and the D-Bar Reconstruction Method

Electrical impedance tomography is a nonlinear inverse problem in which we aim to determine the interior conductivity from current-to-voltage measurements at the boundary. The problem can be formulated as a generalized Laplace equation $$\begin{equation*} \begin{cases} \nabla \cdot \sigma \nabla u &= 0 {~\text {in }}\Omega ,\\ \sigma \mathop {\mathrm {\partial _\nu }}\nolimits u &= \varphi {~\text {on }} \mathop {\mathrm {\partial \Omega }}\nolimits , \\ \end{cases}\tag{1}\end{equation*}$$ modeling the electrical potential $u$ inside the domain $\Omega \subset \mathop {\mathrm {\mathbb R}}\nolimits ^{n}$ for a given conductivity $\sigma$ , with the Neumann boundary condition describing the boundary voltage occurring from the applied mean-free current $\varphi$ . The measurement data consists of pairs of current and voltage measurements and is modeled by the current-to-voltage map $\mathcal {R}_\sigma$ defined by $$\begin{equation*} \mathcal {R}_\sigma \varphi := \left.{ u}\right |_{ \mathop {\mathrm {\partial \Omega }}\nolimits }.\end{equation*}$$ This measurement operator is also known as the Neumann-to-Dirichlet (ND) map, and knowledge of it allows one to predict the resulting voltage for any injected current pattern for $n=2,3$ . In practice, an approximation to the ND map is formed by applying a basis of current patterns and tracking the responses of the voltages. The D-bar algorithm we use below requires the corresponding Dirichlet-to-Neumann (DN) map, which can be obtained as the inverse of the ND map, $\Lambda _{\sigma }=\left ({\mathcal {R}_{\sigma }}\right) ^{-1}$ , for full (vs. partial) boundary data. In this work we consider the $n=2$ case as the D-bar reconstruction framework is further developed in 2D. However, we expect a natural extension to 3D [20].

A. Real-Time Reconstructions Using an Approximate D-Bar Method

By the D-bar method, we refer to the regularized D-bar method [7] based on the theoretical proof given in [21]. The approach uses a nonlinear Fourier transform, called a scattering transform, tailor-made for the EIT problem which is applied to the measured current/voltage data in the form of the DN map $\Lambda _\sigma$ . That scattering data is then used as input data into a partial differential equation, a $\mathop {\mathrm { \mathop {\mathrm {\overline {\partial }}}\nolimits _{k}}}\nolimits$ or ‘D-bar’ equation, giving the method its name. Note that the derivative operators $\mathop {\mathrm { \mathop {\mathrm {\partial }}\nolimits _{z}}}\nolimits$ and $\mathop {\mathrm { \mathop {\mathrm {\overline {\partial }}}\nolimits _{z}}}\nolimits$ are defined as $\mathop {\mathrm { \mathop {\mathrm {\partial }}\nolimits _{z}}}\nolimits = \frac {1}2\left ({\partial _{z_{1}} - i\partial _{z_{2}}}\right)$ and $\mathop {\mathrm { \mathop {\mathrm {\overline {\partial }}}\nolimits _{z}}}\nolimits = \frac {1}2\left ({\partial _{z_{1}} + i\partial _{z_{2}}}\right)$ , where $z=z_{1}+iz_{2}\in \mathop {\mathrm {\mathbb C}}\nolimits$ . The conductivity $\sigma$ is then recovered directly from the solution to the D-bar equation.

The D-bar approach [21] is to transform the physical conductivity equation $\nabla \cdot \sigma \nabla u=0$ into a nonphysical Schrödinger equation, solve that problem instead using the D-bar methods popularized by Beals and Coifman [22], and then transform back to the physical setting. The change of variables $\tilde {u}=\sigma ^{1/2}u$ and $q(z)=\sigma ^{-1/2}(z)\Delta \sigma ^{1/2}(z)$ produces the desired Schrödinger equation $[-\Delta + q(z)]\tilde {u}(z)=0$ , where $z\in \Omega$ . Provided that $\sigma (z)$ is constant in a neighborhood of the boundary, without loss of generality $\sigma =1$ near $\mathop {\mathrm {\partial \Omega }}\nolimits$ , the conductivity can be extended from $\Omega$ to the entire plane by setting $\sigma (z)\equiv 1$ for $z\in \mathop {\mathrm {\mathbb C}}\nolimits \setminus \Omega$ . Note that this gives the potential $q(z)$ compact support in $\Omega$ . We make use of special solutions $\psi (z,k)$ to the Schrödinger equation $$\begin{equation*} [-\Delta + q(z)]\psi (z,k)=0, \quad z\in \mathop {\mathrm {\mathbb C}}\nolimits ,\;\; k\in \mathop {\mathrm {\mathbb C}}\nolimits \setminus \{0\},\tag{2}\end{equation*}$$ called Complex Geometrical Optics (CGO) solutions, that have a specific asymptotic behavior for large $|z|$ or $|k|$ , $\psi (z,k)\sim e^{ikz}$ . Note that we associate $\mathop {\mathrm {\mathbb R}}\nolimits ^{2}$ with $\mathop {\mathrm {\mathbb C}}\nolimits$ via the mapping $z=(z_{1},z_{2})\mapsto z_{1}+iz_{2}$ and thus $kz=\left ({k_{1} + ik_{2}}\right)\left ({z_{1}+iz_{2}}\right)$ denotes complex multiplication. The CGO solutions $\mu (z,k)=e^{-ikz}\psi (z,k)\sim 1$ solve a D-bar equation in the nonphysical scattering variable $k$ $$\begin{equation*} \mathop {\mathrm { \mathop {\mathrm {\overline {\partial }}}\nolimits _{k}}}\nolimits \mu (z,k) = \frac {1}{4\pi \bar {k}} \mathop {{\mathbf {t}}}\nolimits (k)e(z,-k)\overline {\mu (z,k)},\tag{3}\end{equation*}$$ where $e(z,k):=\exp \{i(kz+\bar {k}\bar {z})\}$ and $\mathop {{\mathbf {t}}}\nolimits (k)$ is the nonlinear scattering data defined by $$\begin{equation*} \mathop {{\mathbf {t}}}\nolimits (k):=\int _{ \mathop {\mathrm {\mathbb C}}\nolimits } e(z,k)q(z)\mu (z,k)\;dz.\tag{4}\end{equation*}$$ Note that this scattering data $\mathop {{\mathbf {t}}}\nolimits$ can be thought of as nonlinear Fourier data by the following observation. Replacing the CGO solutions $\mu (z,k)$ in (4) with the asymptotic behavior 1 yields $$\begin{equation*} \mathop {\mathbf {t}^{\text {exp}}}\nolimits (k) = \int _{ \mathop {\mathrm {\mathbb C}}\nolimits } e(z,k)q(z)(1) \;dz = \hat {q}(-2k_{1},2k_{2}),\end{equation*}$$ and thus the ‘Born’ approximation $\mathop {\mathbf {t}^{\text {exp}}}\nolimits$ is essentially a shifted Fourier transform of the potential $q$ . A connection to the measurement data $\Lambda _\sigma$ can be established via Alessandrini’s identity [23] $$\begin{equation*} \mathop {\mathbf {t}^{\text {exp}}}\nolimits (k)=\int _{ \mathop {\mathrm {\mathbb C}}\nolimits } e(z,k) q(z) dz= \int _{ \mathop {\mathrm {\partial \Omega }}\nolimits } e^{i\bar {k}\bar {z}}(\Lambda _\sigma -\Lambda _{1}) e^{ikz} dz.\end{equation*}$$

In this work we use this ‘Born’ approximation $\mathop {\mathbf {t}^{\text {exp}}}\nolimits$ to the scattering data, first presented in [24], as it allows the D-bar method to solve the EIT problem fast enough to be considered ‘real-time’ [25] and is robust against noisy data. The main steps in the algorithm are outlined below:

$$\begin{equation*}\!\!\!\! \begin{array}{c} \text {Current/Voltage Data}\\ \left ({\Lambda _\sigma ,\Lambda _{1}}\right) \end{array} \!\overset {1}{\longrightarrow }\! \begin{array}{c} \text {Scattering Data}\\ \mathop {\mathbf {t}^{\text {exp}}}\nolimits (k) \end{array} \!\overset {2}{\longrightarrow }\! \begin{array}{c} \text {Conductivity}\\ \sigma (z) \end{array}\end{equation*}$$

• Step 1:

  For each $k\in \mathop {\mathrm {\mathbb C}}\nolimits \setminus \{0\}$ , evaluate the approximate scattering data $$\begin{align*} \mathop {\mathbf {t}^{\text {exp}}}\nolimits (k)=\begin{cases} \int _{ \mathop {\mathrm {\partial \Omega }}\nolimits } e^{i\bar {k}\bar {z}}\left ({\Lambda _\sigma \!-\! \Lambda _{1}}\right) e^{ikz} dS(z), & 0<|k|\leq R\\ 0 & |k|>R. \end{cases} \\\tag{5}\end{align*}$$

• Step 2:

  For each $z\in \Omega$ , solve the D-bar equation (3) using the integral equation $$\begin{align*}&\hspace {-1pc} \mathop {\mathrm {\boldsymbol{\mu }^{\text {exp}}}}\nolimits (z,\kappa) \\&= 1+ \frac {1}{4\pi ^{2}}\int _{ \mathop {\mathrm {\mathbb C}}\nolimits }\frac { \mathop {\mathbf {t}^{\text {exp}}}\nolimits (k)e(z,-k)}{(\kappa -k)\bar {k}}\overline { \mathop {\mathrm {\boldsymbol{\mu }^{\text {exp}}}}\nolimits (z,k)}\;d\kappa _{1}d\kappa _{2}, \\\tag{6}\end{align*}$$ and recover the approximate conductivity $$\begin{equation*} \mathop {\mathrm {\sigma ^{\text {exp}}}}\nolimits (z)=\left [{ \mathop {\mathrm {\boldsymbol{\mu }^{\text {exp}}}}\nolimits (z,0)}\right]^{2}.\tag{7}\end{equation*}$$

B. Robustness of D-Bar Methods for EIT

Recent studies [12], [26] suggest that D-bar based reconstruction methods for 2D EIT are robust to incorrect electrode locations and boundary shape. This robustness holds for absolute, as well as time-difference, imaging with both images behaving similarly to incorrect boundary shape and electrode locations. This may be due to the fact that incorrect domain modeling leads to EIT data from a DN map that is only possible for an anisotropic conductivity, even when the true conductivity is isotropic. While the anisotropic conductivity cannot be recovered uniquely, one can recover a unique isotropization, $\sqrt {\det (\sigma)}$ , of the matrix-valued anisotropic conductivity, interpreted as a deformation of the true anisotropic conductivity by isothermal coordinates. In [27] and [28], it is proved that the equations in the D-bar reconstruction methods are identical for anisotropic and isotropic EIT data, helping to explain why D-bar methods have still produced quality images even on anisotropic conductivities and imprecisely known boundary shapes. Here we focus on absolute images.

SECTION III. Deep D-Bar

The aim of this study is to formulate a real-time reconstruction algorithm for electrical impedance tomography that produces sharp and robust absolute EIT images. To achieve this we combine the D-bar algorithm, described in Section II-A, with a convolutional neural network (CNN). This idea relies on a network architecture known as U-Net [29], originally developed for image segmentation. It has been shown for several linear inverse problems [14]–​[18] that this particular network structure can be modified to successfully remove artefacts in medical image reconstructions. The basic recipe is to use a fast and simple reconstruction algorithm to obtain corrupted images and then train the network to remove those artefacts. A related study for electrical impedance tomography is [30], where the authors used artificial neural networks (ANNs) to post-process initial reconstructions from one step of a linear Gauss-Newton algorithm for 3D time-difference EIT imaging. Our approach is fundamentally different as it recovers absolute EIT images.

The network architecture we have chosen relies on the established U-Net [29], which consists of a multilevel decomposition and several skip connections to avoid singularities in the training procedure, see Figure 1 for an illustration of our specific architecture. The original purpose of U-Net was image segmentation. This is very similar to our application, where the main goal is to identify organ boundaries and deconvolve the reconstruction, hence the output of our network is a sharpened image. Therefore, we believe that the U-Net architecture is a suitable choice for the purpose of EIT imaging, since the multilevel structure can deal efficiently with the non-linearity and sharpening over large image areas. Additionally, as discussed in [31], pooling layers leads to translational invariance, which is important to reduce locational bias in the reconstruction process and detect injuries not present in the training set. As a modification to the original architecture we needed to increase the convolutional filter size to $5\times 5$ (compared to $3\times 3$ ), presumably to deal with the nonlinearity of the reconstructions and enforce consistency of the reconstructions. We would like to note, that in contrast to the studies in [14]–​[18], where the authors learn a residual update to the initial reconstruction, we train the network to produce a single sharpened version of the input.

Fig. 1.

Deep D-bar network structure. The input is given by the D-bar reconstruction $\sigma ^{\text {exp}}$ with a resolution of $64\times64$ and the output is denoted by $\widetilde {\sigma }$ . The numbers on top of the blue bars denote the channels for each layer. The resolution for each multilevel decomposition is shown in gray on the left. Each convolutional layer is equipped with a Rectified Linear Unit as nonlinearity, given by ${\sf ReLU{(}x{)}={\sf max} {(}0,x{)}}$ .

A. Training of the Network

Given the true conductivity $\sigma$ , we simulate measurement data, as will be described in Section IV-A, and reconstruct the approximate conductivity $\sigma ^{\exp }$ with the D-bar method outlined in II-A. Since the reconstruction step (6) in the D-bar algorithm can be done for any $z\in \mathop {\mathrm {\mathbb R}}\nolimits ^{2}$ we reconstruct $\sigma ^{\exp }$ on the square $[-1,1]^{2}$ to obtain a square image as input to the network. The resolution is chosen to be $64\times 64$ . The ground truth $\sigma$ is similarly extended to $[-1,1]^{2}$ by extending the background conductivity.

Having obtained the training set $\{\sigma _{i},\sigma _{i}^{\exp }\}_{i}$ , we train the Deep D-bar network, denoted by $\mathcal {D}_\theta$ , for the set of network parameters $\theta$ , i.e. the convolutional filters and biases in each convolutional layer. Given the output of the network $\widetilde {\sigma }=\mathcal {D}_\theta (\sigma ^{\exp })$ we seek to minimize the $\ell ^{2}$ -error of network output to phantom, given by the loss $$\begin{equation*} \mathrm {loss}(\widetilde {\sigma }):=\|\widetilde {\sigma } - \sigma \|_{2}^{2}.\end{equation*}$$ The network is implemented with the Python library Tensor-Flow and the optimization is performed for 1,000 epochs in batches of 16, with TensorFlow’s implementation of the Adam algorithm and an initial learning rate of $10^{-4}$ . The training procedure takes only 4 hours on a single Titan XP GPU with 12GB memory. As we will discuss in the following section, we do not need to perform a transfer training to apply the trained Deep D-bar network to experimental data, the training on simulated data proved to be sufficient.

SECTION IV. Experimental Setup and Computational Notes

We will demonstrate the new Deep D-bar method using experimental data from two different EIT machines: ACT4 [32], [33] from Rensselaer Polytechnic Institute (RPI) as well as KIT4 [34] from the University of Eastern Finland (UEF).

The ACT4 data uses agar (4%) based targets with added graphite (10%) to simulate a heart, two lungs, an aorta, and a spine. All images are shown in DICOM orientation, meaning that the right lung corresponds to the viewer’s left, as if we are looking up through the patient’s feet. Injuries were simulated in the right (DICOM) lung away from the heart by removing a portion of the lung and (1) replacing the missing portion with a piece of agar/graphite with the same conductivity as the heart to simulate an injury such as a pleural effusion, (2) placing three plastic tubes in the missing region to simulate an area of very low conductivity such as a pneumothorax, and (3) replacing the missing portion with three metal tubes. The experiments are shown in Figure 2. The approximate conductivities of the targets are displayed in Table I. The admittivity spectrum of the agar/graphite targets were measured on test-cells with Impedimed’s SFB-7 bioimpedance meter¹. Note that the ACT4 system applies voltages and measures currents rather than vice-versa. In these experiments, trigonometric voltage patterns of maximum amplitude 0.5V (and frequency 3.3kHz) were applied on a circular tank (radius 15cm), with 32 electrodes (width 2.5cm), filled with saline (0.3 S/m) to a height of 2.25cm.

TABLE I Conductivity Values for ACT4 Targets at 3.3kHz

Fig. 2.

Experimental Setups for test phantoms taken on the ACT4 system from RPI. Agar/graphite targets were used to simulate a chest phantom with a heart, two lungs, aorta, and spine. The first image shows the healthy phantom. Three injuries are explored: ‘Injury 1’, replaced the cut portion of the right lung with agar/graphite of the same conductivity as the heart target to simulate a potential pleural effusion, ‘Injury 2’, replaced the cut portion of the right lung replaced with three plastic tubes, and ‘Injury 3’, replaced the cut portion with three copper tubes.

The KIT4 data was taken on a circular tank of radius 14cm with 16 electrodes of width 2.5cm and tap water with conductivity 0.03 S/m filled to a height of 7cm. Conductive (metal) and resistive (plastic) targets were placed in the tank, as shown in Figure 3, and adjacent current patterns with amplitude 2mA were applied at 1kHz. We remark that while this data may not satisfy safety standards for human imaging, it is included for illustrative purposes and potential industrial applications.

Fig. 3.

Experimental Setups with conductive and resistive targets on the KIT4 EIT system from UEF. The white objects are made of solid plastic and are resistive. The hollow circular objects are conductive metal rings.

A. Simulation of 2D EIT Data

The boundary conditions of (1) assume a continuum model for the boundary measurements, completely ignoring the discrete positioning of the electrodes. When simulating the training data, we use a modified version of the continuum model, called the continuum electrode model introduced in [35], which was developed to simulate realistic electrode data in a continuum setting. In essence, the continuum current/voltage traces are optimally projected onto subsets of the boundary corresponding to the electrode locations. The training could be done with a more complicated electrode model, such as the Complete Electrode Model (CEM) [36], however our simplified continuum electrode model proved sufficient for this proof of concept study.

We aim to represent the ND map as matrix approximation $\mathbf {R}_\sigma$ with respect to an orthonormal basis on the boundary. Let $L$ be an even number of electrodes, then the basis functions are chosen for $n\in \{-L/2, {\dots }-1,1, {\dots },L/2\}$ as $$\begin{equation*} \varphi _{n}(\theta)=\begin{cases} \frac {1}{\sqrt {\pi }}\sin (n\theta) & \text {if } n<0,\\ \frac {1}{\sqrt {\pi }}\cos (n\theta) & \text {if } n>0. \end{cases}\end{equation*}$$ The ACT4 system uses $L=32$ electrodes and the KIT4 system uses $L=16$ . The measured voltages are then projected to a continuum trace $g_{n}$ , see [35], [37], and we obtain the ND matrix ${\mathbf {R}}_\sigma$ by evaluating inner products in $L^{2}(\mathop {\mathrm {\partial \Omega }}\nolimits)$ as follows $$\begin{equation*} ({\mathbf {R}}_\sigma)_{n,\ell }=(g_{n},\varphi _\ell)=\int _{ \mathop {\mathrm {\partial \Omega }}\nolimits } g_{n}(s)\varphi _\ell (s) ds.\tag{8}\end{equation*}$$ The matrix approximation of the DN map, $\mathbf {L}_\sigma$ , is then formed by inverting the ND matrix, i.e. $\mathbf {L}_{\sigma }=\left ({\mathbf {R}_{\sigma }}\right)^{-1}$ . If the maximal radius $r$ of the domain is not 1, the DN matrix can be scaled by $r$ to correspond to the data that would be obtained if the radius were 1. Similarly, if $\sigma =\sigma _{0}\neq 1$ near $\mathop {\mathrm {\partial \Omega }}\nolimits$ , the DN matrix is scaled by $\frac {1}{\sigma _{0}}$ to produce the DN matrix that would correspond to $\sigma =1$ near the boundary. If an estimate for $\sigma _{0}$ is not available, the best constant conductivity approximation to the data can be formed as described in [24]. The scaling is undone at the end of the D-bar algorithm by multiplying the conductivity by $\sigma _{0}$ . The matrix approximation $\mathbf {L}_{1}$ to the DN map $\Lambda _{1}$ , required to evaluate the scattering data via (5), is simulated using the constant conductivity $\sigma =1$ .

B. Simulation of Training Data

Training data for the neural network was created using solely simulated data: one group for the ACT4 data and another group for the KIT4 data.

The ACT4 training data was created as follows. Using the ‘Healthy’ image, shown in Figure 2 (top left), approximate organ boundaries were extracted by clicking around the targets in the image for the heart, aorta, left lung, right lung, and spine (Fig. 4, top right). Random numbers were generated to decide whether each individual target was included, heart (95%), aorta (95%), left lung (90%), right lung (90%), spine (100%). If a given target was included, white Gaussian noise (25db) was added to the approximate boundary points of the target using the awgn command in MATLAB to create ‘noisy’ boundary locations. Figure 4 (bottom) shows the effect of the white noise on the boundary locations. Noise was added to each target/organ independently. Conductivities were assigned for each included target by generating a random number from a uniform distribution in the ranges shown in Table I, last column. Elementary injuries were simulated by generating a horizontal dividing line in the lung and assigning randomly generated values in each of the two portions of the divided lung from the uniform distribution of values in [0.01, 1.5], see Fig. 4 bottom right. Each lung had an independent chance of such an injury (30%). More complex injuries could be simulated but are outside the scope of this study. A total of 4,096 simulations were performed for the ACT4 training.

Fig. 4.

Depiction of the simulation of the training data for the ACT4 experiments of Figure 2. The first image shows the healthy phantom from which the ‘true boundary’ (black dots) and ‘approximate boundary’ (red stars) were extracted, shown in the second image. The third and fourth images display sample simulated phantoms using in the training data with and without injuries with the true boundaries overlaid in black dots.

After each conductivity phantom was constructed, the mathematical forward problem (1) was solved to recover the corresponding theoretical boundary voltages and currents using a FEM mesh with 65,536 triangular elements using the continuum electrode model described in Section IV-A. Relative white noise with variance of $10^{-4}$ was added to the measured voltages. The resulting simulated voltages/currents were used to solve the inverse problem using the D-bar method described in Section II-A with a low-pass filtering radius of $R=4.5$ in the scattering domain using the procedure outlined in [38] and uniformly spaced $64 \mathop {\mathrm {\times }}\nolimits 64\,\,k$ and $z$ -grids on $[-4.5,4.5]^{2}$ with stepsize $h_{k}=0.3234$ , and $[-1,1]^{2}$ with stepsize $h_{z}=0.0317$ , respectively. A non-uniform cutoff threshold was enforced on the scattering data for frequencies such that $\mathop {\mathbf {t}^{\text {exp}}}\nolimits (k)=0$ if either $|\Re (\mathop {\mathbf {t}^{\text {exp}}}\nolimits (k)|$ or $|\Im (\mathop {\mathbf {t}^{\text {exp}}}\nolimits (k)|$ exceeds 24. Then, the 4,096 pairs of data in the form of ‘Truth’ and ‘Low-pass D-bar Reconstruction’ were used to train the convolutional neural network described in Section III.

Training data for the KIT4 experiments was simulated in a similar manner. In this case, one to three circular inclusions were simulated, with varying radii drawn from the uniform distribution on [0.2, 0.4], with center in [0, 0.6], and an angle in [0, $2\pi$ ]. Inclusions were not allowed to overlap and each inclusion had an equal probability of being ‘conductive’ or ‘resistive’, and values were assigned from the Uniform distributions [0.05,0.12] and [0.005,0.015] in S/m, respectively. The conductivity of the background was drawn from [0.027, 0.033] S/m. The conductivity ranges for the inclusions were determined from initial higher-pass (larger filtering radius in the scattering domain) D-bar reconstructions of the KIT4 current/voltage data. While we note that the infinite (metal) conductors should have much higher conductivities, this range was observed in the initial testing and proved sufficient for classifying objects as conductors/resistors in our study. Note that such infinite conductors/resistors violate the theory of D-bar which requires inclusions to have non-negative conductivities bounded away from zero and infinity. Nevertheless, the method provides useful conductor/resistor information. The same $k$ and $z$ grids were used in the D-bar reconstructions of the KIT4 example as in the ACT4 example. However, we reduced the non-uniform cutoff threshold of the scattering data from 24 to 8 to reduce oscillatory artefacts that can result from noise in the scattering data for higher frequencies. Figure 5 shows sample phantoms used in the training data for the KIT4 example. A total of 4,096 simulations were performed and pairs of ‘Truth’ vs. ‘Low-pass D-bar Reconstruction’ used to train the network.

Fig. 5.

Depiction of the simulation of the training data for the KIT4 experiments of Figure 3. The images shown are sample simulations of inclusions present in the training data. Zero to three, non-overlapping, circular inclusions were allowed in each simulation.

C. Computational Notes for D-Bar Reconstructions From Experimental EIT Data

The D-bar reconstructions from the experimental ACT4 and KIT4 data were computed in the same manner as the simulated data case described above in Section IV-B with the exception of the formation of the DN matrices $\mathbf {L}_\sigma$ and $\mathbf {L}_{1}$ , which now come from discrete vs. continuous measurements. For convenience, for both the ACT4 and KIT4 data, we synthesized the current/voltage measurements that would have occurred if orthonormal trigonometric current patterns had been applied, by using a change of basis. Define $t^{m}_\ell$ to be the value of the $m$ -th normalized trigonometric current pattern on the $\ell$ -th electrode, following Isaacson et al. [24], $$\begin{align*} t(\ell ,m)=&t^{m}_\ell \\:=&\begin{cases} \sqrt {\frac {2}{L}}\cos (m\theta _\ell), & m=1,\ldots , \frac {L}{2}-1\\ \sqrt {\frac {1}{L}}\cos (m\theta _\ell), & m=\frac {L}{2}\\ \sqrt {\frac {2}{L}}\sin ((m-L/2)\theta _\ell), & m=\frac {L}{2}+1,\ldots , L-1 \end{cases}\normalsize\end{align*}$$ for $\ell =1,\ldots , L$ . This ensures that the matrix of current patterns are orthonormal allowing the solution method of DeAngelo and Mueller [39]. Alternative methods such as Gram-Schmidt based approaches could also be used to produce a matrix of orthonormal currents. The corresponding voltages are formed using a change of basis matrix relating the physical currents and the normalized trig currents. As ACT4 applies voltages and measures currents, we must enforce orthonormality of the currents. Similarly, the KIT4 data used adjacent current patterns which are not orthogonal and must be converted.

Using the approach introduced in [24], the $(m,n)$ entry of the ND matrix $\mathbf {R}_\sigma$ was formed as the discrete inner product $$\begin{equation*} \mathbf {R}_\sigma ~(m,n) = \sum _{\ell =1}^{L} \frac {\phi _\ell ^{m} v^{n}_\ell }{|e_{\ell }|},\quad \begin{array}{l} 1\leq m,n\leq L-1\\ 1\leq \ell \leq L \end{array}\tag{9}\end{equation*}$$ where $\phi ^{m}$ denotes the $m$ -th normalized current pattern, $v^{n}$ the voltage resulting from applying the $m$ -th current pattern (normalized such that the voltages sum to zero and are scaled by the $\ell ^{2}$ norms of the applied current patterns), and $|e_\ell |$ denotes the area of the $\ell$ -th electrode. This formula holds for a system with $L$ electrodes where $L-1$ linearly independent current patterns have been applied.

The discrete DN matrix $\mathbf {L}_\sigma$ was then formed by $\mathbf {L}_{\sigma }=\left ({\mathbf {R}_{\sigma }}\right)^{-1}$ and scaled by $\frac {r}{\sigma _{0}}$ as described above. As the scattering data $\mathop {\mathbf {t}^{\text {exp}}}\nolimits$ (5) requires the difference in DN matrices $\left ({\mathbf {L}_{\sigma }-\mathbf {L}_{1}}\right)$ , the discrete DN matrices $\mathbf {L}_{1}$ must be formed for both the ACT4 and KIT4 system with $L=32$ and $L=16$ electrodes, respectively. To this end, the conductivity equation in (1) was solved, using $\sigma =1$ , with boundary conditions given by the Complete Electrode Model [36] on a FEM mesh with triangular elements (ACT4: 4,149; KIT4: 3,493) simulating injected trigonometric current patterns of amplitude 1mA and non-optimized constant contact impedances of 0.00024 $\Omega \cdot$ mm.

The scattering data $\mathop {\mathbf {t}^{\text {exp}}}\nolimits$ (5) was evaluated as a simple Simpson’s rule approximation, following [39], $$\begin{equation*} \mathop {\mathbf {t}^{\text {exp}}}\nolimits (k) \approx \begin{cases} \frac {2\pi }{L}\left [{e^{i\bar {k}\overline {\mathbf {z}}}}\right]^{T}\phi \left [{\mathbf {L}_\sigma -\mathbf {L}_{1}}\right]\mathbf {e}^ \psi (k) & 0<|k|\leq R\\ 0 & |k|>R \end{cases}\end{equation*}$$ where $\mathbf {z}$ is the vector of the positions of the centers of the electrodes, $T$ denotes the traditional matrix transpose, and $$\begin{equation*}e^ \psi _\ell (k):=\sum _{\ell }^{L} a_{j}(k)\phi ^{j}_\ell \approx {e^{ikz_\ell }}\end{equation*}$$ is the expansion of the asymptotic behavior $\psi \sim e^{ikz}$ at the center of the $\ell$ -th electrode, $z_\ell$ , in the basis of normalized applied current patterns $\phi$ . Then, the d-bar equation (6) can be solved using Fast Fourier Transforms using Vainikko’s method [40]. The interested reader is referred to [38] for further details.

SECTION V. Results

We now demonstrate the effect of the Deep D-bar method on simulated, as well as experimental, data for absolute EIT imaging.

A. Reconstructions From Simulated Data

We begin with purely simulated data for the ACT4 and KIT4 examples. For the ACT4 setting, we consider three scenarios, as shown in Figure 6: one consistent with the training data but not used in the training (top), and two examples violating the training data - one with three horizontally divided regions in the left lung (middle) and the final with a vertical division in the left lung (bottom). Note that the ‘Low-pass D-bar’ and ‘Deep D-bar’ images are shown on the same scale for visual comparison. The complete ‘input’ and ‘output’ of the CNN are on the unit square $[-1,1]^{2}$ . Reconstructions here are clipped to the disc for visualization purposes only. Next, Figure 7 shows the results of the new algorithm on simulated data for the KIT4 example. Three scenarios consistent with the training data, but not used in the training, are presented.

Fig. 6.

Results for simulated test data from the ACT4 geometry. The phantom in the first row conforms with the training data and the phantoms in the second and third row s include pathologies not supported by the training data. The initial D-bar image is compared to the Deep D-bar image. The D-bar images, on the full square are used as the ‘input’ images for the CNN. Images are displayed here on the circular geometry of the tank, for presentation only. Each row is plotted on its own scale.

Fig. 7.

Results for simulated test data from the KIT4 geometry. All phantom are drawn from the same distribution as the training data. The initial D-bar image is compared to the Deep D-bar image. The D-bar images, on the full square are used as the ‘input’ images for the CNN. Images are displayed here on the circular geometry of the tank, for presentation only. Each row is plotted on its own scale.

Structural Similarity Indices (SSIMs) computed for the simulated ACT4 and KIT4 examples are shown in Figures 8 and 9, respectively. Additionally, we evaluated the minimized $\ell ^{2}$ -loss by computing the mean relative error for a test set of 16 samples drawn from the same distribution as the training data. The For the ACT4 simulations we improved from 28.05% to 9.92% and for the KIT4 test data from 16.82% to 9.12% relative $\ell ^{2}$ -error.

Fig. 8.

SSIM measurements are compared for the D-bar method and the new ‘Deep D-bar’ method for the ACT4 reconstructions for the simulated data shown in Figure 6.

Fig. 9.

SSIM measurements are compared for the D-bar method and the new ‘Deep D-bar’ method for the KIT4 reconstructions for the simulated data shown in Figure 7.

B. Reconstructions From Experimental Data

Next, we proceed to reconstructions from experimental data. Figure 10 depicts the results of the Deep D-bar approach on four experiments with ACT4 data: HEALTHY and INJURIES 1-3 as shown in Figure 2. The black dots represent the approximate boundaries of the ‘healthy’ organs, extracted from the photograph. SSIMs (Figure 11) were computed for the experimental reconstructions with the exception of INJURY 3, which has the infinite conductors (copper tubes). The SSIM comparisons used approximate ‘truth’ images formed by assigning the measured conductivity values (Table I) in the respective regions.

Fig. 10.

ACT4 Results for the various test scenarios: Healthy, Injuries 1–3 corresponding to conductive agar, plastic tubes, and conductive copper tubes, respectively. The initial D-bar image is compared to the Deep D-bar image. The D-bar images, on the full square are used as the ‘input’ images for the CNN. Images are displayed here on the circular geometry of the tank, for presentation only. Each row is plotted on its own scale.

Fig. 11.

SSIM measurements are compared for the D-bar method and the new ‘Deep D-bar’ method for the ACT4 experimental data reconstructions shown in Figure 10. Note that meaningful SSIMs could not be computed for ‘Injury 3’ due to the copper inclusions which have infinite conductivity.

Lastly, Figure 12 shows results of the method on the four KIT4 scenarios shown in Figure 3. The overlaid black dots depict the approximate ‘true’ locations of the targets as extracted from their corresponding photographs. No SSIMs were computed here since the objects are infinite conductors and resistors. For a comparison to an iterative method with a total variation prior [9], performed on the same KIT4 data, we refer the reader to the documentation [34].

Fig. 12.

KIT4 Results for the various phantoms with conductive and/or resistive targets, as shown in the first column. The initial D-bar image is compared to the Deep D-bar image. The D-bar images, on the full square are used as the ‘input’ images for the CNN. Images are displayed here on the circular geometry of the tank, for presentation only. Each row is plotted on its own scale.

SECTION VI. Discussion

The reconstructions shown in Figures 6, 7, 10, and 12 demonstrate that Deep D-bar provides superior reconstructions giving both visual and quantitative improvements. In particular, the SSIMs (Figs. 8, 9, and 11) show significant SSIM increases for the Deep D-bar vs. Low-pass D-bar. Note that for the SSIM computation for ACT4 Injury 2 (plastic tubes), the ‘truth’ image was unrealistically set to zero in the lower portion of the right lung, even though the tubes do not entirely fill that region.

We remind the reader that no experimental (truth, reconstruction) pairs were used in training the network and no adaptation to the experimental system was necessary, apart from the number of electrodes in the system. The training was done purely with simulated data. In most applications, either a transfer training [18] or training with a golden standard from the same system must be performed. This demonstrates the robustness of our approach. Additionally, we expect further improvements in the ACT4 reconstructions if more complicated injuries are included in the training and remind the reader that the Low-pass D-bar and Deep D-bar reconstructions are shown on the same scale, which does mask the true dynamic range of the individual images.

We review additional simplifications used in our process: 1) we used the continuum electrode model for the boundary conditions in the training data, 2) the FEM solver used to form $\mathbf {L}_{1}$ for the ACT4 and KIT4 experimental data examples was not finely tuned to either EIT device (which is required for iterative minimization-based methods), and 3) the D-bar solver was not optimized for the respective ACT4/KIT4 data. Rather it was used merely to provide the low-pass reconstructions used as inputs in the CNN. These simplifications were used to demonstrate the robustness of the approach to both noise in the data and tolerance to modeling errors at multiple stages of the reconstruction process.

Initial experiments performed with the original U-Net architecture, i.e. convolutional filters of size $3\times 3$ , did not perform satisfactorily leading us to increase the filter size in this study to $5\times 5$ . This of course leads to an increase in parameters from $8.6\cdot 10^{6}$ to $2.4\cdot 10^{7}$ resulting in longer training times. No batch normalization was needed in our training processes and training times were only 4 hours per network, due to the rather small image size.

The evaluation of the CNN is highly efficient on a GPU and took on average 7.65ms for a single sample, hence we expect Deep D-bar to be real-time capable. This can be done by combining the D-bar reconstruction, as outlined in [25], with the application of the CNN in a unified framework to reduce overhead due to data transmission.

A. Generalization

An important aspect for medical imaging is the robustness and consistency of reconstructions. The successful transition to experimental data suggests that the proposed Deep D-bar method is robust enough for translational imaging. Furthermore, Figures 6 and 10 (ACT4) illustrate that the network can handle reconstructions of phantoms that do not conform with the training data. However, while we were able to localize the inclusions in Figure 12 (Phantom 2.2, KIT4), the sharp angular boundaries of the triangular target were not recovered when using only circular inclusion training data. Our initial testing suggests that this can be improved upon by including significant training on triangular inclusions. Challenges recovering triangular shapes have also been observed in [41]. In terms of image quality, our Deep D-bar approach is comparable to results from (slower) iterative methods on similar data from the KIT4 system, see [9], [34], [41]. Additionally, as the ACT4 injuries we simulated were elementary (only using a horizontal dividing line rather than the true diagonal cut and incomplete regional replacements), we expect that the reconstructions may improve further if more complex injuries were introduced. For human targets, a larger database of training data could be employed and built from an anatomical atlas or collection of CT/MR scans both including and not including abnormalities/injuries.

Crucial for the success of the post-processing network is consistency in the input reconstructions. In order to improve flexibility of the network, one can train the network on reconstructions from scattering data with varying cut-off radii. This allows the user to decide on the quality of the measured data at hand and adjust the cut-off radii as needed for the input reconstruction to the network. First tests have shown that this procedure indeed improves consistency and stability of the reconstructions as illustrated in Figure 13, where we have trained the ACT4 network with varying cut-off radii $R\in [{4,5}]$ . While the SSIMs remained consistent, the localization and recovered conductivity of the injury did improve with new variable radii network.

Fig. 13.

Comparison of results for the ACT4 ‘Injury 1’ (conductive agar in a lung) dataset from two different CNNs. The ‘old’ network denotes the network trained used a fixed cutoff radius for the scattering data ($R =4.5$ ) whereas the ‘new’ network was trained with varying cut-off radii (randomized from the interval [4, 5]). The result from the original, fixed radius $R =4.5$ network, is compared to results using $R =4$ and $R =4.5$ for the input image in the new network. The SSIMs remained consistent: 0.6405, 0.6397, and 0.6459, from left to right.

While we chose, in this study, to match D-bar and CNNs due to the robustness of D-bar for absolute and time-difference imaging and the convolved nature of the D-bar reconstructions, alternative reconstruction methods for the input images could also be used.

SECTION VII. Conclusions

The D-bar method for 2D EIT provides reliable reconstructions of the conductivity but suffers from a blurring due to a low-pass filtering of the scattering data. Sharp improvements in absolute EIT image quality can be achieved by coupling the D-bar reconstruction method with a convolutional neural network. We demonstrated that a CNN can effectively learn the deblurring using only simulated data and still transition to experimental data without including any experimental data in the training itself. As the training can be done offline ahead of time, and the D-bar method provides real-time conductivity reconstructions [25], the post-processing step by the trained CNN adds minimal time to the overall image recovery process, due to the highly efficient evaluation on a GPU. While this work is shown in 2D, we expect the approach to extend to 3D once the D-bar computational framework has been further developed.

ACKNOWLEDGMENTS

The authors gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research. They would like to thank the Isaac Newton Institute for Mathematical Sciences, Cambridge, for support and hospitality during the programme ‘Variational methods and effective algorithms for imaging and vision’ where work on this paper was undertaken. We additionally thank the EIT group at RPI² for their assistance and for providing the ACT4 tank data. Accompanying codes will be made available: https://github.com/asHauptmann/DeepDbar.