Empirically Accelerating Scaled Gradient Projection Using Deep Neural Network for Inverse Problems in Image Processing

Recently, deep neural networks (DNNs) have shown advantages in accelerating optimization algorithms. One approach is to unfold finite number of iterations of conventional optimization algorithms and to learn parameters in the algorithms. However, these are forward methods and are indeed neither iterative nor convergent. Here, we present a novel DNN-based convergent iterative algorithm that accelerates conventional optimization algorithms. We train a DNN to yield parameters in scaled gradient projection method. So far, these parameters have been chosen heuristically, but have shown to be crucial for good empirical performance. In simulation results, the proposed method significantly improves the empirical convergence rate over conventional optimization methods for various large-scale inverse problems in image processing.

1. INTRODUCTION

Inverse problems in image processing applications are often modeled as optimization problems of the form:

$$\begin{equation*}{\min _{\mathbf{x}}}f({\mathbf{x}}) + g({\mathbf{x}})\tag{1}\end{equation*}$$ with forward models f(x) and reasonable image priors g(x). Then, it is optimized using theoretically well-grounded algorithms such as the proximal gradient method (PGM) [1]: $$\begin{equation*}{{\mathbf{x}}^{(k + 1)}} = {\operatorname{prox} _{{t_L}}}(g)\left({{{\mathbf{x}}^{(k)}} - {t_L}\nabla f\left({{{\mathbf{x}}^{(k)}}}\right)}\right)\tag{2}\end{equation*}$$ where denotes l₂-norm, and 1/t_L is larger than the Lipschitz constant of f(x). Many related works have been proposed to improve theoretical convergence rates of the algorithm or to design good priors to regularize ill-posed inverse problems [2], [3], [4].