Multi-Contrast Brain MRI Image Super-Resolution With Gradient-Guided Edge Enhancement

In magnetic resonance imaging (MRI), the super-resolution technology has played a great role in improving image quality. The aim of this paper is to improve edges of brain MRI by incorporating the gradient information of another contrast high-resolution image. Multi-contrast images are assumed to possess the same gradient direction in a local pattern. We proposed to establish a relation model of gradient value between different contrast images to restore a high-resolution image from its input low-resolution version. The similarity of image patches is employed to estimate intensity parameters, leading a more accurate reconstructed image. Then, an iterative back-projection filter is applied to the reconstructed image to further increase the image quality. The new approach is verified on synthetic and real brain MRI images and achieves higher visual quality and higher objective quality criteria than the compared state-of-the-art super-resolution approaches. The gradient information of the multi-contrast MRI images is very useful. With a proper relation model, the proposed method enhances image edges in MRI image super-resolution. Improving the MRI image resolution from very low-resolution observations is challenging. We tackle this problem by first modeling the relation of gradient value in multi-contrast MRI and then performing fast supper-resolution methods. This relation model may be helpful for other MRI reconstruction problems.

SECTION I. Introduction

In magnetic resonance imaging (MRI), the low-resolution (LR) is usually encountered due to acquisition constrains such as limited sampling time or moving subjects. These LR may seriously affect the post-processing and medical diagnosis. To recover a high-resolution (HR) image from its low-resolution version [1], super-resolution technologies have been widely used in MRI [2]–​[11].

Super-resolution methods can be roughly grouped into two main categories: Interpolation and non-interpolation. Interpolation approaches, e.g., the bicubic and bi-spline, are usually fast but frequently generate over-smooth images. Non-interpolation methods incorporate various image priors, e.g., sparse representation [3]–​[6], [11] non-local reconstruction [7], [10] and total variations [8], [9], leading to more attractive super-resolved images. However, these non-interpolation methods usually require time-consuming iteration processing and do not pay special attention to edge information.

Edge structures of medical images have a particularly significant impact on visual scenes to detect suspicions, classify malformations and make diagnosis [15]. Thus, it is valuable to consider edge factor in image super-resolution. For example, edges are preserved pretty well in a contrast-guided interpolation (CGI) approach [16]. Edges are also sharpened by considering gradient features [2], [12]. These methods, only explore the information from a single target image in the low resolution, thus a faithful super-resolution may be hard to achieve since the available information is very limited.

Other prior information beyond a single image is expected to improve the image resolution [5], [7], [13], [14], [17], [18]. For example, non-local similarity of image patches is introduced as the prior information [5], [7], [13], [14], [17]. These methods also require time-consuming iteration processing. In addition, Zheng et al. [18] introduced a fast interpolation approach using the local weight similarity (ILWS) between multi-contrast brain images to improve the resolution of the LR MRI image. But, it does not perform satisfactorily when the LR image is too smooth.

Recently, an effective interpolation method that establishing gradient relations between the LR image and HR reference image with different contrast was proposed [19] in the remote sensing. This method has a simple but effective non-iterative linear model, and does not require training sets. Image edges were interpolated much well according to the reported results [19]. This model is called as the Super-Resolution of using Gradient Relations (SRGR) model here. The SRGR provides a promising direction to model relationship between the multi-contrast images due to its fast computing and attractive performance. The SRGR, as far as we know, has not been applied for MRI. What is more, to fit for the multi-contrast MRI interpolation, SRGR should be carefully modified since the images in MRI are very different from those in remote sensing [19]. This aim of this work is to model the gradient relations of multi-contrast MRI images and improve MRI super-resolution.

In this work, we propose a new gradient linear relation model for multi-contrast brain MRI super-resolution, which is inspired by SRGR. The gradient information on edges of a HR reference is used to guide the interpolation of a LR MRI image. To further improve the resolution, the non-local similarity of image patches is employed to robustly estimate the intensity parameter of the model and the Iterative Back-Projection (IBP) filter is enforced. Experimental results will verify more promising results for our method than the compared MRI interpolation approaches.

This paper is structured as follows. Section II gives a brief review of the related work. Section III presents the proposed approach. Section IV provides experimental results followed by discussions in Section V. Finally, we summarize this work in Section VI.

SECTION II. Background and Related Works

A. Multi-Contrast Images in MRI

Multi-contrast images are frequently acquired in MRI [1], [20] and the commonly acquired ones are T1w and T2w. Plentiful edge structures are visible in these two contrast images of the same subject (Fig. 1). According to the MRI physics [21], the image $S\left ({{\vec {r}} }\right)$ of T1w or T2w is generated as follows:

$$\begin{equation*} S\left ({{\vec {r}} }\right)\propto \rho \left ({{\vec {r}} }\right)\left ({{1-e^{-TR/T_{1} \left ({{\vec {r}} }\right)}} }\right)\left ({{e^{-TE/T_{2} \left ({{\vec {r}} }\right)}} }\right),\tag{1}\end{equation*}$$ where $\rho \left ({{\vec {r}} }\right)$ is the proton density at spatial location $\vec {r}$ , $TR$ refers to the repetition time and $TE$ denotes the echo time. By setting different values of $TR$ and $TE$ , multi-contrast images will be acquired. Yet, these images share the proton density of the subject and hence they largely share similar anatomical structures but with different contrasts in regions. The shared information between inter-contrast images was previously considered to profit the super-resolution [18] and other image reconstruction tasks [22]–​[24].

FIGURE 1.

Multi-contrast MRI brain images: (a) a T1w image; (b) a T2w image.

B. Brief Reviews of SRGR Model

The SRGR model was proposed in super-resolution of remote sensing images [19]. Its basic idea is described below.

Suppose that a LR image $\tilde {{{{\mathbf{X}}}}}_{L}$ of its target HR image $\tilde {{{{\mathbf{X}}}}}$ (Fig. 2(a)) and a HR reference image $\tilde {{{{\mathbf{R}}}}}$ (Fig. 2(b)) are available while $\tilde {{{{\mathbf{X}}}}}$ and $\tilde {{{{\mathbf{R}}}}}$ have same size. SRGR consists two steps: 1) A pre-interpolated image ${{\mathbf{X}}}$ (Fig. 2(c)), that is with the same size of $\tilde {{{{\mathbf{X}}}}}$ , is obtained with some classic interpolation methods; 2) Gradient information from $\tilde {{{{\mathbf{R}}}}}$ is applied to update X.

FIGURE 2.

Illustration for SRGR model in a local pattern. (a) is the target HR image $\tilde {{{{\mathrm{X}}}}}$ ; (b) is the HR reference image $\tilde {{{{\mathrm{R}}}}}$ with different contrast; (c) is the pre-interpolated image X obtained from the low observation of $\tilde {{{{\mathrm{X}}}}}$ ; (d) An example of the gradient direction $l$ in a local pattern and the edge direction. Note: The gradient direction is denoted by the vector $l$ . In estimation, a more accurate pixel, e.g. $X_{i,j} +\delta _{i,j}$ , indicated by dark circles within (c) is used to replace the pre-interpolated HR pixel, e.g. $X_{i,j}$ , to generate a better HR image.

To improve the interpolation, the same gradient direction is assumed between the multi-contrast images $\tilde {{{{\mathbf{X}}}}}$ and $\tilde {{{{\mathbf{R}}}}}$ . The relationship of gradient is modeled as [19]

$$\begin{equation*} \frac {g(\tilde {X}_{i,j})}{g(\tilde {R}_{i,j})}=\lambda ^{0},\tag{2}\end{equation*}$$ where $g\left ({\cdot }\right)$ is a second order gradient function, $\tilde {X}_{i,j}$ is the pixel of $\tilde {{{{\mathbf{X}}}}}$ , $\tilde {R}_{i,j}$ is the pixel of $\tilde {{{{\mathbf{R}}}}}$ and $\lambda ^{0}$ is a parameter that models this relationship ideally.

Assuming that an adjustment parameter $\delta _{i,j}$ is added to the ($i$ , $j$ ) pixel $X_{i,j}$ of a pre-interpolated image ${{\mathbf{X}}}$ , the SRGR model is formulated as [19]

$$\begin{align*}&\hspace {-1pc}\min \limits _{\delta _{i,j}} \{[g(X_{i,j} +\delta _{i,j})-\lambda g(\tilde {R}_{i,j})]^{2} \\&\qquad \qquad \qquad ~\,\, \quad +\,[g_{\bot } (X_{i,j} +\delta _{i,j})-\lambda _{\bot } g_{\bot } (\tilde {R}_{i,j})]^{2}\}\tag{3}\end{align*}$$ by considering a gradient direction $l$ (Fig. 2(d)) and its normal direction, also called edge direction, $l_{\bot }$ (Fig. 2(d)).

Now, we are prepared to discuss the essential parts in Eq. (3). First, the gradient functions $g(X_{i,j} +\delta _{i,j})$ and $g_{\bot } (X_{i,j} +\delta _{i,j})$ are defined as

$$\begin{align*} g(X_{i,j} +\delta _{i,j})=&X_{i+1,j-1} +X_{i-1,j+1} -2(X_{i,j} +\delta _{i,j}) \\ g_{\bot } (X_{i,j} +\delta _{i,j})=&X_{i-1,j-1} +X_{i+1,j+1} -2(X_{i,j} +\delta _{i,j})\tag{4}\end{align*}$$ where $X_{i+1,j-1}$ and $X_{i-1,j+1}$ are the neighborhood pixels of $X_{i,j}$ along the direction $l$ , while $X_{i-1,j-1}$ and $X_{i+1,j+1}$ are the nearest neighbors of $X_{i,j}$ along the direction $l_{\bot }$ , respectively. Second, the $\lambda$ and $\lambda _{\bot }$ in Eq. (3) are parameters that adjusting intensity in two directions. Too smaller $\lambda$ and $\lambda _{\bot }$ will lead to blur edges. On the contrary, too larger $\lambda$ and $\lambda ^{\bot }$ will cause artificial edges [19]. In SRGR, their values are estimated as $$\begin{align*} \lambda=&(X_{i+1,j-1} -X_{i-1,j+1})/(\tilde {R}_{i+1,j-1} -\tilde {R}_{i-1,j+1}) \\ \lambda _{\bot }=&(X_{i-1,j-1} -X_{i+1,j+1})/(\tilde {R}_{i-1,j-1} -\tilde {R}_{i+1,j+1}).\tag{5}\end{align*}$$

Overall, the SRGR model in Eq. (3) tries to refine the pixels of pre-interpolated image by minimizing the consistent second order gradient that is parallel and perpendicular to the edges. Since each term in this model is quadratic, Eq. (3) can be solved fast by forcing the first derivative of the objective function to be zero. More details of implementation can be found in [19].

However, the basic SRGR may not suit well for recovering MRI since the local-contrast may be more complex than the remote sensing images presented in [19]. How to modify the SRGR to fit for multi-contrast MRI will be the main focus of our work.

SECTION III. Proposed Method

An overview of the proposed approach is summarized in Fig. 3. First, the LR image is pre-interpolated by the CGI method. Second, the HR reference image is registered to pre-interpolated image (Benefit of registration is shown in Appendix C). Then, the HR image of interest will be restored by using the gradient information of the registered HR reference image in another contrast. Finally, the HR image will be enhanced with an IBP filter. In the following, the essential parts of the proposed method will be presented.

FIGURE 3.

Block diagram of the proposed method.

A. Measuring Gradient Difference on Same Contrast Images

The essence of SRGR model in Eq. (3) is to estimate the relationship of the second gradient order between the pre-interpolated image ${{\mathbf{X}}}$ and the HR reference image $\tilde {{{{\mathbf{R}}}}}$ . Now, let us consider an ideal step. Assume that an ideal pre-interpolation can generate a HR image ${{\mathbf{X}}}$ that is as good as the target HR image $\tilde {{{{\mathbf{X}}}}}$ . Accordingly, there may exist a pre-interpolated reference image ${{\mathbf{R}}}$ that is as good as $\tilde {{{{\mathbf{R}}}}}$ . Then, according to Eq. (2), the following relationship holds true

$$\begin{equation*} \frac {g(X_{i,j})}{g(R_{i,j})}\approx \lambda ^{0},\tag{6}\end{equation*}$$ where $R_{i,j}$ is the ($i$ , $j$ ) pixel of ${{\mathbf{R}}}$ . Through the elementary mathematical manipulation of Eqs. (2) and (6), one obtains $$\begin{equation*} \lambda ^{0^\prime }=\frac {g(\tilde {X}_{i,j})-g(X_{i,j})}{g(\tilde {R}_{i,j})-g(R_{i,j})}\approx \lambda ^{0},\tag{7}\end{equation*}$$ which is the ratio of gradient difference estimated from the same contrast images.

The Eq. (7) implies that one may turn to measure the ratio of the gradient difference for the pairs of target/pre-interpolated images and reference/pre-interpolated reference images. This modification is important since the gradient difference is computed within one contrast, but not across two contrasts in the original SRGR in Eq. (2). Thus, Eq. (7) is able to reduce the effect of large contrast variations on degrading the multi-contrast MRI super-resolution, which will be systematically analyzed in the following section.

B. Super-Resolution Model

Based on Eq. (7), our model can be expressed as

$$\begin{align*}&\hspace {-1pc}\min \limits _{\delta _{i,j}} \{ [(g(X_{i,j} +\delta _{i,j})-g(X_{i,j}))-\lambda (g(\tilde {R}_{i,j})-g(R_{i,j}))]^{2} \\&\,\,+[(g_{\bot } (X_{i,j} +\delta _{i,j})-g_{\bot } (X_{i,j})) \\&\,\,-\lambda _{\bot } (g_{\bot } (\tilde {R}_{i,j})-g_{\bot } (R_{i,j}))]^{2}\}\tag{8}\end{align*}$$ which finds refinements on pixels to minimize the gradient difference between the HR images and HR reference images.

In this model, the accuracy of $\lambda$ and $\lambda _{\bot }$ is essential for image reconstruction. Thus, error analysis on $\lambda$ (same analysis for $\lambda _{\bot }$ is omitted) and edge pixels will be briefly given below (More details can be found in Appendix A). The following analysis will show that, comparing with the SRGR, the new method will obtain lower reconstruction error on edge pixels.

In our model, the error on $\lambda$ is denoted by $\Delta \lambda ^{\prime }$ defined as

$$\begin{equation*} \Delta \lambda ^{\prime }=\lambda ^{0^\prime }-\lambda,\tag{9}\end{equation*}$$ and the reconstruction error on edge pixels is denoted by $\Delta {\delta }'_{i,j}$ . According to Eq. (8), we know that $$\begin{equation*} \Delta {\delta }'_{i,j} \propto \Delta \lambda ^{\prime }[g(\tilde {R}_{i,j})-g(R_{i,j})].\tag{10}\end{equation*}$$

In SRGR model (Eq. (3)), the error $\Delta \lambda$ is

$$\begin{equation*} \Delta \lambda =\lambda ^{0} -\lambda.\tag{11}\end{equation*}$$ and its reconstruction error $\Delta \delta _{i,j}$ is determined as $$\begin{equation*} \Delta \delta _{i,j} \propto \Delta \lambda g(\tilde {R}_{i,j}).\tag{12}\end{equation*}$$

Then, since $\lambda ^{0^\prime }\approx \lambda ^{0}$ in Eq. (7), combing Eqs. (9) and (11) we will get

$$\begin{equation*} \Delta \lambda \approx \Delta \lambda ^{\prime }.\tag{13}\end{equation*}$$

In addition, most edge pixels in the HR reference image $\tilde {R}_{i,j}$ and the pre-interpolated reference image $R_{i,j}$ satisfy (More details are provided in Appendix A)

$$\begin{equation*} \left |{ {g(\tilde {R}_{i,j})-g(R_{i,j})} }\right | < \left |{ {g(\tilde {R}_{i,j})} }\right |.\tag{14}\end{equation*}$$ Finally, by comparing Eq. (10) with Eq. (12), it can be concluded that the condition $$\begin{equation*} \left |{ {\Delta {\delta }'_{i,j}} }\right | < \left |{ {\Delta \delta _{i,j}} }\right |\tag{15}\end{equation*}$$ holds for most edge pixels. This indicates, in our model, the reconstruction error of edge pixels is mostly smaller than that of SRGR.

Above analysis can be verified in MRI images with smooth pixels at the boundary of two contrast regions, such as the laminar structure [7] in brain. The brain MRI image possesses the characteristics of laminar structure which is formed by the brain gyri and sulci [7]. In Figs. 4(a) and 4(b), one pair of multi-contrast toy examples which imitate local laminar structures is presented. With arbitrarily changing local contrasts, both Eqs. (14) and (15) are often satisfied as shown in Figs. 4(c) and 4(d), respectively. Analysis on more complex images, e.g. Brainweb MRI [25], is given in Fig. 5. The average reconstruction error is measured as the root mean square error between the reconstructed pixel value and the original pixel value. Fig. 5(e) implies that Eq. (15) also holds true (More details of Figs.4 and 5 are presented in Appendix B).

FIGURE 4.

Error analysis on a toy example with the characteristics of laminar structure. (a) is the HR original image, in which $p_{1} =0$ , $q_{1} =1$ and the edge pixel value is the average of $p_{1}$ and $q_{1}$ ; (b) is the HR reference image, in which $p_{2} =1$ , $q_{2} =0$ and the edge pixel value changes from 0 to 1. This change of edge pixel value is embodied on the horizontal axis of Fig. 3(c-d). (c) Curves for $\left |{ {g(\tilde {R}_{i,j})} }\right |$ and $\left |{ {g(\tilde {R}_{i,j})-g(R_{i,j})} }\right |$ of the edge pixel. (d) Curves for reconstruction errors.

FIGURE 5.

Comparison of reconstruction errors on Brainweb MRI data. (a) HR image using SRGR model, the PSNR = 21.54; (b) HR image using the proposed model, the PSNR = 24.17; (c) original T2w image (ground truth); (d) original T1w image (HR reference). (e) Mean error of reconstructed HR pixel values versus second order gradients.

C. Robust Estimation for $\lambda$ and $\lambda_{\bot}$

We further propose a robust estimation for $\lambda$ (the same method for $\lambda _{\bot }$ ). We assume that $\lambda$ is the same for all similar patches within a local region $\Omega$ . First, we search similar patches in $\Omega$ using a fast global search method that exploring the same geometric direction for patches is adopted [22], [26], [27]. Then, the $\lambda$ is computed for each patch according to Eq. (5). Last, the $\lambda$ for all similar patches are averaged to a final $\lambda _{\Omega }$ for these patches. Improvement on estimating $\lambda$ with similar patches is shown in Fig. 6, indicating that $\lambda _{\Omega }$ successfully avoids abnormal-pixel-values encountered using different $\lambda$ for different patches.

FIGURE 6.

Reconstruction comparisons of $\lambda$ and $\lambda _{\Omega }$ on Brainweb MRI data. (a) HR image produced with using $\lambda$ ; (b) HR image using the proposed model (i.e., $\lambda _{\Omega }$ are exploited), the PSNR = 23.85. Note: the ground truth and HR reference images are presented in Fig. 5(c) and (d), respectively.

D. Image Improvement With IBP

As we discussed before, the CGI method is utilized to perform the initial interpolation on the original LR image to produce the pre-interpolated image. Edge pixels in the pre-interpolated image are reconstructed one by one (in the order: from top left to bottom right) by our model and replace original residents of the pre-interpolated image. Finally, the HR image with more sharp and distinct edges will be obtained.

In order to take into account MRI acquisition properties, we impose Iterative Back-Projection (IBP) on the reconstructed HR pixels. The IBP iteratively minimizes the differences between the original LR image and LR version of the reconstructed HR image. Previously, in MRI super-resolution field, IBP was used for recovering image details in LR brain images [6], [28].

The IBP process is given by

$$\begin{equation*} ({{\mathbf{X}}}+\boldsymbol {\delta })^{(t+1)}=({{\mathbf{X}}}+{\boldsymbol {\delta }})^{(t)}+({{\mathbf{U}}}(\tilde {{{{\mathbf{X}}}}}_{L} -{{\mathbf{DS}}}({{\mathbf{X}}}+{\boldsymbol {\delta }})^{(t)})){{\mathbf{P}}}\tag{16}\end{equation*}$$ where $({{\mathbf{X}}}+\boldsymbol {\delta })^{(t)}$ denotes the estimated HR image after the $t^{\textrm {th}}$ iterative calculation, and $\tilde {{{{\mathbf{X}}}}}_{L}$ denotes the original LR image. P is the back project filter, S is the blurring filter, D is the down-sampling operator and U is the up-sampling operator whose factor is the same with D. More details and analysis about IBP can be found in [29].

For MRI images, IBP will improve the PSNR (Fig. 7(a)) of reconstruction by increasing the number of iterations (denoted by $k$ ) in the IBP. Considering the computation complexity, we choose $k=5$ for all experiments in this paper, which generally yields satisfactory results (Fig. 7(d)).

FIGURE 7.

Magnified regions of reconstruction results versus the number of iterations $k$ . (a) The PSNRs of using IBP versus the different $k$ . The bottom row shows a zoom-up region in the reconstruction result and from left to right: (b) $k=0$ (i.e., without IBP); (c) $k=1$ ; (d) $k=5$ .

SECTION IV. Results

In this section, the proposed approach will be compared with the bicubic interpolation, the state-of-the-art CGI [16] and the ILWS that considers the multi-contrast images [18]. Experiments are conducted on Brainweb brain images and real MRI images. All methods run on a personal computer with Dual-Core CPU 3.00GHz and 2 GB memory.

Two objective criteria, the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) [30], are exploited to quantitatively evaluate the super-resolution results. The higher PSNR indicates that the reconstructed image is more consistent to the original HR image and the higher SSIM implies that more accurate image structures are preserved.

A. Simulation Set Up

Before conducting reconstructions, a LR image is generated from an original HR image by using a Gaussian smooth filter and a down sampling operator. This is a common way of simulating LR images in MRI super-resolution [6]. In experiments, the Gaussian smooth filter is in the size of $3\times 3$ with standard deviation of 1 and the down-sampling factor of 2. The LR image is interpolated by CGI because CGI takes an important factor, edge contrast, into interpolation and effectively avoid image artifacts with fairly low computational complexity.

The purpose of our method is to recover edge details of LR brain image. In our work, a pixel is declared to be an edge pixel if the local variance within its nearest neighbors is greater than or equal to an established threshold of 0.0003, on the premise that intensities of images are all normalized between 0 and 1. The same value of the threshold is utilized in all experiments of this paper.

B. Comparisons With Other Methods

Brainweb [25] (http://www.bic.mni.mcgill.ca/brainweb/) is a publicly available dataset which provides two kinds of MRI simulated image: normal (e.g., Imag_B1 and Imag_B2) and multiple sclerosis (e.g., Imag_B3 and Imag_B4). A note about multiple sclerosis lesions data is that they have been extracted from real MRI data. The data of in-plane pixel size is automatically set as $1\times 1$ mm. In addition, we select noise percentages of 1%. The original 2D T1w and T2w data are in the size of $181 \times 217$ .

We also assess the performance of our algorithm on real brain MRI images which have four types:

1) Imaging Data Shared by Philips Company

Imag_H1 and Imag_H2 (noting: test images of TABLE 1 in the revision are numbered) are shared by Philips Company. The T1w (TR = 170 ms, TE = 3.9ms) and T2w (TR = 3000ms, TE = 80ms) images of $256\times 256$ size are acquired with fast field echo sequence ($\text {FOV}=230\times 230$ mm2, slice thickness = 5.0 mm).

TABLE 1 PSNR/SSIM Evaluation for Defferent Methods

2) Imaging Data From Siemens Scanners

Imag_P1 and Imag_P2 are acquired from a 3T Siemens Trio Tim MRI scanner using a turbo spin echo sequence ($\text {FOV}=230\times 187$ mm2, slice thickness = 5.0mm) and the matrix size of original T1w (TR = 2000ms, TE = 9.7ms) and T2w (TR = 5000ms, TE = 97ms) images is $384\times 324$ .

3) Public NAMIC Data

Imag_N1 and Imag_N2 are acquired from – a publicly available MRI brain dataset (http://hdl.handle.net/1926/1687), which is provided by National Alliance for Medical Image Computing (NAMIC). The dataset is acquired at a 3T GE scanner at BWH in Boston, MA, and contains 10 normal persons and 10 schizophrenic patients. The matrix size of the T1w (TR = 7.4 ms, TE = 3 ms) or T2w (TR = 2500 ms, TE = 80 ms) data is $256\times 256\times 176$ , with voxel resolution of $1\times 1\times 1$ mm3. In this paper, two 2D brain images are acquired from extracting 3D data along the coronal plane.

4) Imaging Data From East China Normal University

Imag_E1 and Imag_E2 are from East China Normal University dataset (ECNU_set). The experiments were carried out on a 3 T MRI scanner (Siemens Trio Tim), with FOV of $256\times 256$ mm2. The acquired dataset includes T1w (TR = 440 ms, TE = 2.46 ms) and T2w (TR = 6750 ms, TE = 90 ms) images with 3mm slice, and also T1w (TR = 250 ms, TE = 2.46 ms) and T2w (TR = 4500 ms, TE = 90 ms) images with 4 mm slice. All images are with pixel size of $256\times 256$ . The T1w and T2w data are acquired with gradient echo pulse sequence and turbo spin echo sequence, respectively.

these 1), 2), and 4) studies were respectively approved by Institutional Review Board of Philips Company, Shenzhen Institutes of Advanced Technology, and East China Normal University. Their proper informed consents were obtained from all volunteers prior to enrollment.

To better display edge structures in the experimental results, we provide zoom-in images (Fig. 8). Among them, reconstructions of normal and multiple sclerosis anatomical datasets are depicted in Figs. 8 (a) and (b), respectively. Results on real datasets are presented in Figs. 8 (c)-(e). From top to bottom are the result of the bicubic, CGI, ILWS, the proposed method without the IBP, the proposed method with the IBP, the ground truth image and HR reference image. Full figures are given in the appendix C.

FIGURE 8.

Magnified sections of reconstructed images. Among them, (a-b) are respectively results of Imag_B1 and Imag_B3 from the Brainweb dataset; (c-e) are based on the real data and are results of Imag_H1, Imag_P1 and Imag_P2. From top to bottom: Bicubic; CGI; ILWS; the proposed method without IBP; the proposed method with IBP; the ground truth image; the HR reference image.

In Fig. 8, the bicubic method ($1^{\mathrm {st}}$ row) results in more blur structure details than other methods. ILWS ($3^{\mathrm {rd}}$ row) achieves sharper edges compared to the bicubic, but the ringing artifacts can still be observed along the edges. Furthermore, CGI ($2^{\mathrm {nd}}$ row) recovers edges better than ILWS. Obviously, the proposed algorithm without IBP ($4^{\mathrm {th}}$ row) delivers more subjective results with sharp edges than CGI. In addition, details of edge structures are further clearly revealed in results of the proposed method with IBP ($5^{\mathrm {th}}$ row).

TABLE 1 summarizes the PSNR and SSIM values for image interpolation. It shows that ILWS obtains higher PSNR and SSIM than classic bicubic, but CGI outperforms both Bicubic and ILWS. The proposed method, however without IBP or with IBP, outperforms the other three methods.

SECTION V. Discussions

In this section, we first evaluate the effect of noise on the proposed method, and then extend the comparison with a state-of-the art non-local MRI upsampling techniques, and finally report the the computational time of different methods.

A. Effects of Noise

The noise at common levels (0%, 1%, 3% and 5% of the maximum intensity) [31], [32] are added into the T2-weighted ground truth image. To simulate the Rician noise in real data (Imag_H1, Imag_P1, Imag_N1, and Imag_E1) [31], [32], the zero mean Gaussian noise are added to real and imaginary parts of ground truth images, respectively.

Results on different level noise (Figs. 9 (a) and (b)) imply that increasing the noise level will reduce the PSNR and SSIM. These two criteria are seriously reduced when the noise level is equal or greater than 3%, indicating obvious degraded images in Figs. 9 (f) and (g).

FIGURE 9.

Effects of noise on the proposed method. (a) and (b) are PSNR and SSIM performance. Noting: Noise are randomly generated for 30 times and the PSNR and SSIM are averaged over the 30 super-resolution experiments. Visual results based Imag_B1with different noise level are presented in (c-g): 0%, 1%, 3% and 5% noise, respectively. Note: IBP is imbedded in the proposed method.

B. Comparison With Non-Local Upsampling

we add the comparison with the method “Non-local MRI upsampling” [10], which is referred as NMU here. Since NMU is a super-resolution algorithm for degraded 3D MRI images with average filtering, we rewrite it into a 2D algorithm with Gaussian filtering for facilitating the comparison.

Results in Figs. 10 (a) and (b) show that the NMU obtains good PSNR and SSIM values and proposed method achieves better evaluation criteria than the NMU. To further verify the performance, super-resolved images are reported in Figs. 10 (e) and (f), indicating that the proposed method provides more consistent image structures to the ground-truth image shown in (Fig. 10 (d)).

FIGURE 10.

Comparison with NMU. (a) and (b) are PSNR and SSIM performance, respectively. Visual results based Imag_E1 are presented in (c-f): HR reference image, ground truth image, NMU, and proposed method. Note: CGI-based pre-interpolation and IBP are imbedded in the proposed method.

It is worth noting that, NMU performs very well on the degraded images with average filtering, which is the problem originally discussed in [10], as shown in TABLE 2. With the NMU is as the pre-interpolation, the proposed method can still improve the super-resolved images.

TABLE 2 Reconstruction Performance (PSNR/SSIM) on Degraded Images With Average Filtering

C. Computation Time

The computation time of different methods are shown in TABLE 3. Traditional methods consume no more than 2 seconds while the proposed method requires 35 seconds. Thus, the new method is relatively time consuming.

TABLE 3 Computation Time for Defferent Methods (Units: Seconds)

SECTION VI. Conclusion

We proposed a new super-resolution method for MRI multi-contrast images by using gradient information of the high-resolution reference image that has another contrast. The connection of gradient value between different MRI contrast images is studied and an associated model is established to estimate values of edge pixels inside the pre-interpolated image. Experimental results demonstrate that the proposed algorithm is effective for both synthetic and real MRI images. The IBP filter is suggested to be applied after our algorithm to further improve the quality of super-resolved MRI images.

In this work, the proposed method is mainly developed for 2D MRI images. Extension and application of this algorithm on 3D data will be an interesting future work. In addition, how to retrieve structural distinctions hidden in multi-contrast MRI images more accurately still need further investigation. Last, this modeling may benefit improving reconstruction of fast imaging in MRI [22], [33]–​[38] and reconstruction of multi-contrast MRI images in fast sampling [39]–​[41].

ACKNOWLEDGMENTS

The authors sincerely thank Dr. Kai-Kuang Ma for sharing CGI code, Dr. Kaihua Zhang for providing the data in Fig.C5 and Kourosh Jafari-Khouzani for valuable suggestions. They also appreciate reviewers for their constructive comments.