Corrections to “On the Contractivity of Plug-and-Play Operators”

Presents corrections to the article “On the Contractivity of Plug-and-Play Operators”.

In this note, we rectify mistakes in the proof of [1, Lemma 1] and the statement of [1, Th. 2]. The “necessity” part in the proof of [1, Lemma 1] is incorrect, which is rectified below.

Proof of [1, Lemma 1]:

$(\Leftarrow)$: See the supplement of [1].

$(\Rightarrow)$: Suppose there exists $\mathbf {0}\ne \boldsymbol{x}\in \mathbb {U}\cap \mathbb {V}$. Since $\boldsymbol{x}\in \mathbb {V}$, it follows from the definition of $\mathbb {V}$ that $\boldsymbol{x}= \boldsymbol{y}+\boldsymbol{z}$ where $\boldsymbol{y}\in \mathcal {N}(\mathbf {I}-\mathbf {N})$ and $\boldsymbol{z}\in \mathcal {N}(\mathbf {I}+\mathbf {N})$; furthermore, as $\mathbf {N}\in \mathbb {S}^{n}$, we have $\Vert \boldsymbol{x}\Vert ^{2}=\Vert \boldsymbol{y}\Vert _{2}^{2} + \Vert \boldsymbol{z}\Vert _{2}^{2}$. Let $\boldsymbol{u}:= \boldsymbol{y}-\boldsymbol{z}$; notice that $\mathbf {N}\boldsymbol{u}=\boldsymbol{x}$. Moreover, since $\mathbf {N}\boldsymbol{u}\in \mathbb {U}$ and $\Vert \mathbf {M}\boldsymbol{v}\Vert _{2} = \Vert \boldsymbol{v}\Vert _{2}$ for all $\boldsymbol{v}\in \mathbb {U}$, we get $$\begin{equation*} \Vert \mathbf {M}(\mathbf {N}\boldsymbol{u})\Vert _{2}^{2} = \Vert \mathbf {N}\boldsymbol{u}\Vert _{2}^{2} = \Vert \boldsymbol{x}\Vert _{2}^{2} = \Vert \boldsymbol{y}\Vert _{2}^{2} + \Vert \boldsymbol{z}\Vert _{2}^{2} = \Vert \boldsymbol{u}\Vert _{2}^{2}, \end{equation*}$$ This contradicts $\Vert \mathbf {M}\mathbf {N}\Vert _{2} < 1$.

Theorem 1 ([1, Th. 2]):

Let $\mathbf {W}$ be a symmetric denoiser and $\rho > 0$. Then $\Vert \mathbf {R}\Vert _{2} < 1$ if and only if $(\mathcal {N}(\mathbf {I}-\mathbf {W}) \oplus \mathcal {N}(\mathbf {W})) \cap \, \mathcal {N}(\mathbf {A}) = \lbrace \mathbf {0}\rbrace$.

For the ease of reference, we recall that [1, Eq. (8)] $$\begin{align*} \mathbf {R}=& \frac{1}{2}(\mathbf {I}+ \mathbf {J}), \quad \mathbf {J}= \mathbf {F}\mathbf {V}, \tag{1a}\\ \mathbf {F}=& 2(\mathbf {I}+ \rho \mathbf {A}^\top \mathbf {A}) ^{-1} - \mathbf {I},\quad \mathbf {V}= (2\mathbf {W}-\mathbf {I}). \tag{1b} \end{align*}$$ It can be checked (using [1, Lemma 1]) that $(\mathcal {N}(\mathbf {I}-\mathbf {W}) \oplus \mathcal {N}(\mathbf {W})) \cap \mathcal {N}(\mathbf {A}) = \lbrace \mathbf {0}\rbrace$ is equivalent to $\Vert \mathbf {J}\Vert _{2} < 1$. However, $\Vert \mathbf {J}\Vert _{2} < 1$ is sufficient but not necessary for $\Vert \mathbf {R}\Vert _{2} < 1$. Thus, the condition in Theorem 1 is only sufficient for $\Vert \mathbf {R}\Vert _{2} < 1$; the correct statement of Theorem 1 is as follows.

Theorem 2:

Let $\mathbf {W}$ be a symmetric denoiser and $\rho > 0$. If $(\mathcal {N}(\mathbf {I}-\mathbf {W}) \oplus \mathcal {N}(\mathbf {W})) \cap \, \mathcal {N}(\mathbf {A}) = \lbrace \mathbf {0}\rbrace$, then $\Vert \mathbf {R}\Vert _{2} < 1$.

We next give an equivalent condition for $\Vert \mathbf {R}\Vert _{2} < 1$.

Theorem 3:

Let $\mathbf {W}$ be a symmetric denoiser and $\rho > 0$. Then $\Vert \mathbf {R}\Vert _{2} < 1$ if and only if $\mathcal {N}(\mathbf {I}-\mathbf {W}) \cap \mathcal {N}(\mathbf {A}) = \lbrace \mathbf {0}\rbrace$.

Remark 1:

The condition in Theorem 3 is weaker than the sufficient condition in Theorem 2. In fact, the condition in Theorem 3 is identical to the equivalent condition in [1, Th. 1] for the contractivity of PnP-ISTA operator.$\hfill\blacklozenge$

In the light of Theorem 3, we can drop the nonsingularity assumption on $\mathbf {W}$ in [1, Corollary 2] to obtain the following stronger result; see [1, Sec. III-A] for more details.

Corollary 1:

Let $\mathbf {W}$ be a symmetric denoiser for which 1 is a simple eigenvalue. Then, for any $\rho > 0$, PnP-ADMM is contractive and converges linearly for inpainting, deblurring, and superresolution.

We now prove Theorem 3, starting with some preliminaries. We recall the result [1, Proposition A.1] from the supplement.

Proposition 1:

Let $\mathbf {T}\in \mathbb {S}^{n}$ be such that $\Vert \mathbf {T}\Vert _{2} \leqslant 1$. Let $\mathbb {Y}\subseteq \mathbb {R}^{n}$ be the space spanned by eigenvectors of $\mathbf {T}$ corresponding to eigenvalues $\pm 1$. Then $\Vert \mathbf {T}\boldsymbol{z}\Vert _{2} < \Vert \boldsymbol{z}\Vert _{2}$ for all $\boldsymbol{z}\in \mathbb {R}^{n} \setminus \mathbb {Y}$.

Remark 2:

Notice from (1b) that $\mathbf {F}\in \mathbb {S}^{n}$. We can conclude using the spectral mapping theorem that $\sigma (\mathbf {F}) \subseteq (-1, 1]$ for all $\rho > 0$, where $\sigma (\mathbf {F})$ is the spectrum of $\mathbf {F}$; consequently, $\Vert \mathbf {F}\Vert _{2} \leqslant 1$. Also, since $\sigma (\mathbf {W}) \subseteq [0,1]$ and $\mathbf {W}\in \mathbb {S}^{n}$, we have $\sigma (\mathbf {V}) \subseteq [-1, 1]$ and $\Vert \mathbf {V}\Vert _{2} \leqslant 1$.$\hfill\blacklozenge$

Remark 3:

In the context of Proposition 1, the corresponding subspaces for $\mathbf {F}$ and $\mathbf {V}$ are $\mathcal {N}(\mathbf {A})$ and $\mathcal {N}(\mathbf {I}-\mathbf {W}) \oplus \mathcal {N}(\mathbf {W})$, respectively. Thus, $\Vert \mathbf {F}\boldsymbol{z}\Vert _{2} < \Vert \boldsymbol{z}\Vert _{2}$ for all $\boldsymbol{z}\in \mathbb {R}^{n} \setminus \mathcal {N}(\mathbf {A})$, and $\Vert \mathbf {V}\boldsymbol{z}\Vert _{2} < \Vert \boldsymbol{z}\Vert _{2}$ for all $\boldsymbol{z}\in \mathbb {R}^{n} \setminus (\mathcal {N}(\mathbf {I}-\mathbf {W}) \oplus \mathcal {N}(\mathbf {W}))$. $\hfill\blacklozenge$

Proof of Theorem 3:

$(\Rightarrow)$: Suppose there exists $\mathbf {0}\ne \boldsymbol{x}\in \mathcal {N}(\mathbf {I}-\mathbf {W}) \cap \mathcal {N}(\mathbf {A})$. From $\boldsymbol{x}\in \mathcal {N}(\mathbf {I}-\mathbf {W})$, we get $\mathbf {V}\boldsymbol{x}= \boldsymbol{x}$; and from $\boldsymbol{x}\in \mathcal {N}(\mathbf {A})$, we have $\mathbf {F}\boldsymbol{x}= \boldsymbol{x}$. Consequently, using (1), we get $\mathbf {J}\boldsymbol{x}=\boldsymbol{x}$ and $\mathbf {R}\boldsymbol{x}=\boldsymbol{x}$, which contradicts $\Vert \mathbf {R}\Vert _{2} < 1$.

$(\Leftarrow)$: Let $\mathbb {W}:= \mathcal {N}(\mathbf {I}-\mathbf {W}) \oplus \mathcal {N}(\mathbf {W})$. For any $\boldsymbol{x}\in \mathbb {R}^{n} \setminus \lbrace \mathbf {0}\rbrace$, we consider the following exhaustive cases.

• Case $(\boldsymbol{x}\notin \mathbb {W}).$ It follows from Remarks 2 and 3 that $$\begin{equation*} \Vert \mathbf {J}\boldsymbol{x}\Vert _{2} \leqslant \Vert \mathbf {V}\boldsymbol{x}\Vert _{2} < \Vert \boldsymbol{x}\Vert _{2}. \end{equation*}$$ Therefore, using (1), $\Vert \mathbf {R}\boldsymbol{x}\Vert _{2} < \Vert \boldsymbol{x}\Vert _{2}$.

• Case $(\boldsymbol{x}\in \mathbb {W}).$ This means $\boldsymbol{x}=\boldsymbol{y}+\boldsymbol{z}$ where $\boldsymbol{y}\in \mathcal {N}(\mathbf {I}-\mathbf {W})$ and $\boldsymbol{z}\in \mathcal {N}(\mathbf {W})$. There are two subcases: (i) $\mathbf {V}\boldsymbol{x}\notin \mathcal {N}(\mathbf {A})$, and (ii) $\mathbf {V}\boldsymbol{x}\in \mathcal {N}(\mathbf {A})$. In subcase-(i), it follows from (1) and Remark 3 that $\Vert \mathbf {J}\boldsymbol{x}\Vert _{2} < \Vert \mathbf {V}\boldsymbol{x}\Vert _{2} \leqslant \Vert \boldsymbol{x}\Vert _{2}$; thus, $\Vert \mathbf {R}\boldsymbol{x}\Vert _{2} < \Vert \boldsymbol{x}\Vert _{2}$.

  In subcase-(ii), since $\mathcal {N}(\mathbf {I}-\mathbf {W}) \cap \mathcal {N}(\mathbf {A}) = \lbrace \mathbf {0}\rbrace$, we claim that $\boldsymbol{z}\ne \mathbf {0}$. Note that $\mathbf {V}\boldsymbol{x}= \boldsymbol{y}-\boldsymbol{z}$. Thus, if $\boldsymbol{z}= \mathbf {0}$, we have $\mathbf {V}\boldsymbol{x}=\boldsymbol{y}\in \mathcal {N}(\mathbf {I}-\mathbf {W}) \cap \mathcal {N}(\mathbf {A})$ which contradicts $\mathcal {N}(\mathbf {I}-\mathbf {W}) \cap \mathcal {N}(\mathbf {A}) = \lbrace \mathbf {0}\rbrace$. Notice that since $\mathbf {V}\boldsymbol{x}\in \mathcal {N}(\mathbf {A})$, we have $\mathbf {F}(\mathbf {V}\boldsymbol{x})=\mathbf {V}\boldsymbol{x}$. Subsequently, using (1) and $\mathbf {V}\boldsymbol{x}= \boldsymbol{y}-\boldsymbol{z}$, we get $$\begin{equation*} \mathbf {R}\boldsymbol{x}= \frac{1}{2}(\boldsymbol{x}+ \mathbf {J}\boldsymbol{x}) = \frac{1}{2}(\boldsymbol{x}+ \mathbf {V}\boldsymbol{x}) = \boldsymbol{y}. \tag{2} \end{equation*}$$ Since $\mathbf {W}\in \mathbb {S}^{n}$, we have $\Vert \boldsymbol{x}\Vert ^{2}=\Vert \boldsymbol{y}\Vert _{2}^{2} + \Vert \boldsymbol{z}\Vert _{2}^{2}$. Therefore, it follows from (2) and $\boldsymbol{z}\ne \mathbf {0}$ that $\Vert \mathbf {R}\boldsymbol{x}\Vert _{2} < \Vert \boldsymbol{x}\Vert _{2}$.

We have shown that $\Vert \mathbf {R}\boldsymbol{x}\Vert _{2} < \Vert \boldsymbol{x}\Vert _{2}$ for all $\boldsymbol{x}\in \mathbb {R}^{n} \setminus \lbrace \mathbf {0}\rbrace$; hence, $\Vert \mathbf {R}\Vert _{2} < 1$.$\blacksquare$