A Symmetric Bregman Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problem

The alternating direction method of multipliers (ADMM) and its symmetric version are efficient for minimizing two-block separable problems with linear constraints. However, both ADMM and symmetric ADMM have limited versatility across various fields due to the requirement that the gradients of differentiable functions exhibit global Lipschitz continuity, a condition that is typically challenging to satisfy in nonconvex optimization problems. Recently, a novel Bregman ADMM that not only eliminates the need for global Lipschitz continuity of the gradient, but also ensures that Bregman ADMM can be degenerated to the classical ADMM has been proposed for two-block nonconvex optimization problems with linear constraints. Building on this, we propose a symmetric Bregman alternating direction method of multipliers, which can be degenerated into the symmetric ADMM and the Bregman ADMM, and thus further degenerated into the classical ADMM. Moreover, when solving separable nonconvex optimization problems, it does not require the global Lipschitz continuity of the gradients of differentiable functions. Furthermore, we demonstrate that under the Kurdyka-Lojasiewicz inequality and certain conditions, the iterative sequence generated by our algorithm converges to a critical point of the problem. In addition, we examine the convergence rate of the algorithm.

• Symmetric ADMM
• Bregman ADMM
• Nonconvex Optimization
• Separable
• Kurdyka-Lojasiewicz Inequality

1. Introduction

In this paper, we consider the following two-block separable optimization problem with linear constraint

$\begin{array}{ll}\underset{x,y}{\text{min}}\hfill & f\left(x\right)+g\left(y\right),\hfill \\ \text{s}\text{.t}.\hfill & Ax+y=b,\hfill \end{array}$(1.1)

where $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is a proper lower semicontinuous function, $g:{ℝ}^m\to ℝ$ is a continuous differentiable function, $A\in {ℝ}^{m\times n}$ and $b\in {ℝ}^m$. Numerous valuable optimization problems can be expressed in the form (1), rendering it widely applicable across diverse fields, such as machine learning [1] - [3] , image processing [4] - [6] , and signal processing [7] - [10] .

When both $f$ and $g$ are convex functions, a prominent approach for addressing problem (1) is the alternating direction method of multipliers (ADMM), proposed by Gabay, Mercier, Glowinski and Marroco [11] [12] in 1970s. This method fully harnesses the separable properties to their utmost potential, thus attracting considerable attention across diverse domains in recent years. The iterative scheme of the ADMM is as follows

$\{\begin{array}{l}{x}_{k+1}\in \underset{x}{\arg \min }\left\{{ℒ}_{\beta }\left(x,y_k,{\lambda }_k\right)\right\},\\ y_{k+1}\in \underset{y}{\arg \min }\left\{{ℒ}_{\beta }\left(x_{k+1},y,{\lambda }_k\right)\right\},\\ {\lambda }_{k+1}:={\lambda }_k-\beta \left(A{x}_{k+1}+y_{k+1}-b\right).\end{array}$(1.2)

Here, ${ℒ}_{\beta }(\cdot )$ denotes the augmented Lagrangian function for (1.1), given by

${ℒ}_{\beta }\left(x,y,\lambda \right)=f\left(x\right)+g\left(y\right)-\langle \lambda ,Ax+y-b\rangle +\frac{\beta }{2}{\Vert Ax+y-b\Vert }^2,$

where $\lambda$ is the Lagrangian multiplier associated with the linear constraint, and $\beta >0$ is the penalty parameter. The study of ADMM has a lengthy academic lineage, its convergence and convergence rate are well-understood [13] - [16] for convex objectives.

However, for scenarios where there is at least one nonconvex component in the objective function, many studies primarily focus on proving the convergence of the ADMM or its variant and analyzing problem scenarios under additional conditions, such as Li and Pong [17] and Hong et al. [18] . Particularly, in 2017, Guo et al. [19] demonstrated that under conditions less stringent than those outlined in [17] [18] , the convergence and convergence rate of ADMM for nonconvex problems can be established, contingent upon the augmented Lagrangian function satisfying the Kurdyka-Lojasiewicz inequality. Motivated by the insights provided in the aforementioned article, Wu et al. [20] considered utilizing symmetric ADMM to address a two-block linearly constrained separable nonconvex optimization, which can revert back to the classical ADMM, and conducted an analysis on its convergence and convergence rate for the case $B=I$ ( $I$ is the identity matrix with proper dimension). Note that it can numerically accelerate ADMM with some values of $\alpha >0$. Its iterative format is as follows

$\{\begin{array}{l}{x}_{k+1}\in \underset{x}{\arg \min }\left\{{ℒ}_{\beta }\left(x,y_k,{\lambda }_k\right)\right\},\\ {\lambda }_{k+\frac{1}{2}}:={\lambda }_k-\alpha \beta \left(A{x}_{k+1}+B{y}_k-b\right),\\ y_{k+1}\in \underset{y}{\arg \min }\left\{{ℒ}_{\beta }\left(x_{k+1},y,{\lambda }_{k+\frac{1}{2}}\right)\right\},\\ {\lambda }_{k+1}:={\lambda }_{k+\frac{1}{2}}-\beta \left(A{x}_{k+1}+B{y}_{k+1}-b\right),\end{array}$(1.3)

The only difference from the classical ADMM lies in the addition of a relaxation factor $\alpha \in \left(-1,1\right)$ in the update of the multiplier ${\lambda }^{k+1}$ between the iterative formulas of $x$ and $y$. In particular, the algorithm returns to the classical ADMM when $\alpha =0$. Therefore, it presents the same limitations in certain areas [21] - [23] . Whether using the ADMM or the symmetric ADMM to tackle two-block separable nonconvex problems with linear constraints, both methods are constrained by the assumption of Lipschitz continuity of differentiable functions.

To relax the Lipschitz continuity constraint on the gradient of the objective function, Tan and Guo [24] introduced a novel version of Bregman ADMM, which is distinguished from that proposed by Wang et al. [25] by its ability to revert to the classical ADMM. Its iteration is as follows

$\{\begin{array}{l}{x}_{k+1}\in \underset{x}{\arg \min }\left\{{ℒ}_{\beta }^h\left(x,y_k,{\lambda }_k\right)\right\},\\ y_{k+1}\in \underset{y}{\arg \min }\left\{{ℒ}_{\beta }^h\left(x_{k+1},y,{\lambda }_k\right)\right\},\\ {\lambda }_{k+1}:={\lambda }_k-\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right),\end{array}$(1.4)

where ${ℒ}_{\beta }^h(\cdot )$ denotes the Bregman augmented Lagrangian function for (1.1)

${ℒ}_{\beta }^h\left(x,y,\lambda \right)=f\left(x\right)+g\left(y\right)-\langle \lambda ,Ax+y-b\rangle +\beta D_h\left(-y,Ax-b\right),$(1.5)

where $\lambda$ is the Lagrangian multiplier associated with the linear constraint, and $\beta >0$ is the penalty parameter. And the Bregman distance $D_h\left(\cdot ,\cdot \right)$ is defined as

$D_h\left(x,y\right)=h\left(x\right)-h\left(y\right)-\nabla h{\left(y\right)}^{\text{T}}\left(x-y\right).$

When $h(\cdot )=\frac{1}{2}{\Vert \cdot \Vert }^2$, the Bregman ADMM (1.4) reduces to the classical ADMM (1.2).

Drawing on the aforementioned concept, our aim is to alleviate the necessity for Lipschitz continuity of the gradient of differentiable functions in the symmetric ADMM while addressing two-block separable nonconvex problems with linear constraints. In this paper, we propose an iteration for symmetric version of the Bregman ADMM, whose iterative scheme is

$\{\begin{array}{ll}{x}_{k+1}\in \underset{x}{\arg \min }\left\{{ℒ}_{\beta }^h\left(x,y_k,{\lambda }_k\right)\right\},\hfill & (1.6\text{a})\hfill \\ {\lambda }_{k+\frac{1}{2}}:={\lambda }_k-\alpha \beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right),\hfill & (1.6\text{b})\hfill \\ y_{k+1}\in \underset{y}{\arg \min }\left\{{ℒ}_{\beta }^h\left(x_{k+1},y,{\lambda }_{k+\frac{1}{2}}\right)\right\},\hfill & (1.6\text{c})\hfill \\ {\lambda }_{k+1}:={\lambda }_{k+\frac{1}{2}}-\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right),\hfill & (1.6\text{d})\hfill \end{array}$

The symmetric Bregman ADMM (1.6) can transition to the symmetric ADMM (1.3) by setting $h(\cdot )=\frac{1}{2}{\Vert \cdot \Vert }^2$, to the Bregman ADMM (1.4) by setting $\alpha =0$, and to the classic ADMM (1.2) by setting $h(\cdot )=\frac{1}{2}{\Vert \cdot \Vert }^2$ and $\alpha =0$. In essence, the ADMM framework investigated by Wu et al. [20] and Tan and Guo [24] is a particular instance of the approach we introduce. Under the same assumption as those in Tan and Guo [24] , we can prove the convergence of the symmetric Bregman ADMM (1.6), providing that the associated function satisfies the Kurdyka-Lojasiewicz inequality. Moreover, we demonstrate that the iterative sequence produced by the symmetric Bregman ADMM (1.6) converges to a critical point of the problem (1.1), and we also analyze the convergence rate of the algorithm.

The remainder of this paper is organized as follows. In Section 2, we provide some necessary preliminaries for our subsequent analysis. In Section 3, we present the convergence analysis of the symmetric Bregman ADMM (1.6) and analyze its convergence rate. Finally, in Section 4, we summarize our findings and draw conclusions.

2. Preliminaries

In this section, we recall some definitions and basic results that will be used for further analysis.

Definition 2.1. [26] For an extended real-valued function $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$, the effective domain or just the domain is the set

$dom\left(f\right)=\left\{x\in {ℝ}^n:f\left(x\right)<\infty \right\}.$

Definition 2.2. [26] A function $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is called proper if there exists at least one $x\in {ℝ}^n$ such that $f\left(x\right)<\infty$.

Definition 2.3. [26] A function $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is called lower semicontinuous at $x\in {ℝ}^n$ if

$f\left(x\right)\le \underset{k\to \infty }{\lim \inf }f\left(x_k\right).$

for any sequence $\left\{x_k\right\}\subseteq {ℝ}^n$ for which $x_k\to x$ as $k\to \infty$. Moreover, $f(\cdot )$ is called lower semicontinuous if it is lower semicontinuous at each point in ${ℝ}^n$.

Definition 2.4. ( [27] , kernel generating distance) Let $C$ be a nonempty, convex, and open subset of ${ℝ}^m$. Associated with $C$, a function $h:{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$ is called a kernel generating distance if it satisfies the following

(i) $h(\cdot )$ is proper, lower semicontinuous, and convex, with $dom\left(h\right)\subset \bar{C}$ and $dom\left(\partial h\right)=C$.

(ii) $h(\cdot )$ is $C^1$ on $int\left(dom\left(h\right)\right)\equiv C$.

We denote the class of kernel generating distance by $\mathcal{G}\left(C\right)$.

Definition 2.5 ( [27] , L-smooth adaptable) Let $h\in \mathcal{G}\left(C\right)$, $g:{ℝ}^m\to ℝ$ continuously differentiable on $C=int\left(dom\left(h\right)\right)$. A pair $\left(g,h\right)$ is called L-smooth adaptable on $C$ if there exists $L>0$ such that $Lh-g$ and $Lh+g$ are convex on $C$.

Remark 2.1. Definition 2.5 serves as a natural extension and complement to the definition of “A Lipschitz-like/Convexity Condition” as presented in reference [21] . This extension enables the derivation of the following two-sided descent lemma.

Lemma 2.1. ( [27] , extended descent lemma) The pair of functions $\left(g,h\right)$ is L-smooth adaptable on $C$ if and only if

$\left|g\left(x\right)-g\left(y\right)-\langle \nabla g\left(y\right),x-y\rangle \right|\le L{D}_h\left(x,y\right)\forall x,y\in int\left(dom\left(h\right)\right).$(2.1)

Remark 2.2. In particular, when the set $C={ℝ}^m$ and $h(\cdot )=\frac{1}{2}{\Vert \cdot \Vert }^2$, (2.1) reduces to the classical descent lemma for function $g$, i.e.,

$\left|g\left(x\right)-g\left(y\right)-\langle \nabla g\left(y\right),x-y\rangle \right|\le \frac{L}{2}\cdot {\Vert x-y\Vert }^2\forall x,y\in {ℝ}^m.$

Definition 2.6. [27] Let $g:{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$ be a proper and lower semicontinuous function. The gradient of $g(\cdot )$ is D-Lipschitz if there exists $L>0$ satisfying

$\Vert \nabla g\left(x\right)-\nabla g\left(y\right)\Vert \le L\frac{D_h\left(x,y\right)+D_h\left(y,x\right)}{\Vert x-y\Vert },x\ne y\in int\left(dom\left(h\right)\right).$

Remark 2.3. According to Cauchy-Schward inequality, we have

$\left|\langle \nabla g\left(x\right)-\nabla g\left(y\right),x-y\rangle \right|\le \Vert \nabla g\left(x\right)-\nabla g\left(y\right)\Vert \Vert x-y\Vert ,$

which combines with definition ??, we obtain that

$\left|\langle \nabla g\left(x\right)-\nabla g\left(y\right),x-y\rangle \right|\le L\left(D_h\left(x,y\right)+D_h\left(y,x\right)\right).$

Using the conclusion in Lemma 2.4, the above inequality is equivalent to

$\langle \nabla g\left(x\right)-\nabla g\left(y\right),x-y\rangle \le L\left(D_h\left(x,y\right)+D_h\left(y,x\right)\right)=L\langle \nabla h\left(x\right)-\nabla h\left(y\right),x-y\rangle ,$

$\langle \nabla g\left(y\right)-\nabla g\left(x\right),x-y\rangle \le L\left(D_h\left(x,y\right)+D_h\left(y,x\right)\right)=L\langle \nabla h\left(x\right)-\nabla h\left(y\right),x-y\rangle .$

Then,

$\langle \left(L\nabla h\left(x\right)-\nabla g\left(x\right)\right)-\left(L\nabla h\left(y\right)-\nabla g\left(y\right)\right),x-y\rangle \ge 0,$

$\langle \left(L\nabla h\left(x\right)+\nabla g\left(x\right)\right)-\left(L\nabla h\left(y\right)+\nabla g\left(y\right)\right),x-y\rangle \ge 0.$(2.2)

Based on inequality (2.2), it can be concluded that the functions $Lh+g$ and $Lh-g$ exhibit convexity due to the monotonicity of their gradients on the set $C$. Hence, ensuring the gradient of function $g(\cdot )$ satisfies the D-Lipschitz continuity condition is adequate for establishing the $\left(g,h\right)$ function pair as L-smooth adaptable. Considering the intricacy of the ADMM iterative procedure in this study, we find it necessary to presuppose the D-Lipschitz continuity of function $g(\cdot )$.

Remark 2.4. Certainly, the D-Lipschitz continuity characteristic is essentially a Lipschitz-like gradient property in the context of the Bregman distance for the function $g(\cdot )$, and it becomes equivalent to the gradient Lipschitz continuity of $g(\cdot )$ when $h(\cdot )=\frac{1}{2}{\Vert \cdot \Vert }^2$.

Definition 2.7. [27] Let $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper lower semicontinuous function.

(i) The Fréchet subdifferential, or regular subdifferential, of $f(\cdot )$ at $x\in dom\left(f\right)$, written $\hat{\partial }f\left(x\right)$, is the set of vectors $x^{\text{*}}\in {ℝ}^n$ that satisfy

$\underset{y\ne x}{\text{lim}}\underset{y\to x}{\text{inf}}\frac{f\left(y\right)-f\left(x\right)-\langle x^{\text{*}},y-x\rangle }{\Vert y-x\Vert }\ge 0.$

When $x\notin dom\left(f\right)$, we set $\hat{\partial }f\left(x\right)=\varnothing$.

(ii) The limiting-subdifferential, or simply the subdifferential, of $f(\cdot )$ at $x\in dom\left(f\right)$, written $\partial f\left(x\right)$, is defined as follows:

$\partial f\left(x\right)=\left\{x^{\text{*}}\in {ℝ}^n,\exists x_n\to x,f\left(x_n\right)\to f\left(x\right),x_n^{\text{*}}\in \hat{\partial }f\left(x_n\right)\text{with}{x}_n^{\text{*}}\to x^{\text{*}}\right\}.$

Remark 2.5. From above definition, we note that

(i) It implies $\hat{\partial }f\left(x\right)\subseteq \partial f\left(x\right)$ for each $x\in {ℝ}^n$, where the first set is closed convex while the second one is only closed.

(ii) Let $\left(x_k,x_k^{\text{*}}\right)\in Graph\partial f$ be a sequence that converges to $\left(x,x^{\text{*}}\right)$. By the definition of $\partial f$, if $f\left(x_k\right)$ converges to $f\left(x\right)$ as $k\to +\infty$, then $\left(x,x^{\text{*}}\right)\in Graph\partial f$, where $Graph\partial f=\left\{\left(x,y\right)|y\in \partial f\left(x\right)\right\}$.

(iii) A necessary condition for $x\in {ℝ}^n$ to be a minimizer of $f(\cdot )$ is

$0\in \partial f\left(x\right).$(2.3)

(iv) If $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is a proper lower semicontinuous function and $g:{ℝ}^n\to ℝ$ is continuous differentiable function, then $\partial \left(f+g\right)\left(x\right)=\partial f\left(x\right)+\nabla g\left(x\right)$ for any $x\in dom\left(f\right)$.

A point that meets the condition of Equation (2.3) is referred to as a critical or a stationary point. The critical points set of $f$ is denoted by $\text{crit}f$.

Next, we recall an important property of subdifferential calculus.

Lemma 2.2. [28] Suppose that $F\left(x,y\right)=f\left(x\right)+g\left(x\right)$, where $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ and $g:{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$ are proper lower semicontinuous functions. Then for all $\left(x,y\right)\in dom\left(F\right)=dom\left(f\right)\times dom\left(g\right)$, we have

$\partial F\left(x,y\right)={\partial }_{x}F\left(x,y\right)\times {\partial }_{y}F\left(x,y\right).$

Definition 2.8. ( [28] , Kurdyka-Lojasiewicz inequality) Let $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper lower semicontinuous function. For $-\infty <{\eta }_1<{\eta }_1\le +\infty$, set

$\left[{\eta }_1<f<{\eta }_1\right]=\left\{x\in {ℝ}^n:{\eta }_1<f\left(x\right)<{\eta }_2\right\}.$

We say that function $f(\cdot )$ has the KL property at $x^{\text{*}}\in dom\left(\partial f\right)$ if there exist $\eta \in \left(0,+\infty \right]$, a neighbourhood $U$ of $x^{\text{*}}$, and a continuous concave function $\phi :\left[0,\eta \right)\to {ℝ}_{+}$, such that

(i) $\phi \left(0\right)=0$;

(ii) $\phi$ is $C^1$ on $\left(0,\eta \right)$ and continuous at 0;

(iii) ${\phi }^{\prime }\left(x\right)>0$, $\forall x\in \left(0,\eta \right)$;

(iv) for all $x$ in $U\cap \left[f\left(x^{\text{*}}\right)<f<f\left(x^{\text{*}}\right)+\eta \right]$, the Kurdyka-Lojasiewicz inequality holds

${\phi }^{\prime }\left(f\left(x\right)-f\left(x^{\text{*}}\right)\right)d\left(0,\partial f\left(x\right)\right)\ge 1,$

where $d\left(x,\partial f\left(x\right)\right)=\underset{y\in \partial f\left(x\right)}{\text{inf}}\Vert y-x\Vert$, is the distance from $x$ to $\partial f\left(x\right)$.

Remark 2.6. Denote ${\Phi }_{\eta }$ be the set of all continuous functions $\phi (\cdot )$ which satisfy (i)-(iii).

Definition 2.9. ( [29] , Kurdyka-Lojasiewicz function) If function $f(\cdot )$ satisfies the KL property at each point of $dom\left(\partial f\right)$, then $f(\cdot )$ is called a KL function.

Lemma 2.3. ( [30] , Uniformized KL property) Let $\Omega$ be a compact set and $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper and lower semicontinuous function. Assume that $f(\cdot )$ is constant on $\Omega$ and satisfies the KL property at each point of $\Omega$. Then, there exist $\epsilon >0$, $\eta >0$, and $\phi \in {\Phi }_{\eta }$ such that for all $\bar{x}\in \Omega$ and for all $x$ in the following intersection:

$\left\{x\in {ℝ}^n:d\left(x,\Omega \right)<\epsilon \right\}\cap \left[f\left(\bar{x}\right)<f<f\left(\bar{x}\right)+\eta \right].$

one has

$\phi \text{'}\left(f\left(x\right)-f\left(\bar{x}\right)\right)d\left(0,\partial f\left(x\right)\right)\ge 1.$

Lemma 2.4. [31] Let $h:{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$. For any $x,y\in int\left(dom\left(h\right)\right)$ and $z\in dom\left(h\right)$, then

(i) $D_h\left(x,y\right)+D_h\left(y,x\right)=\langle \nabla h\left(x\right)-\nabla h\left(y\right),x-y\rangle$.

(ii) Three points identity holds:

$D_h\left(z,x\right)-D_h\left(z,y\right)-D_h\left(y,x\right)=\langle \nabla h\left(x\right)-\nabla h\left(y\right),y-z\rangle .$(2.4)

Definition 2.10. [32] Let $h\in \mathcal{G}\left(C\right)$. The Bregman distance $D_h:dom\left(h\right)\times int\left(dom\left(h\right)\right)\to {ℝ}^{+}$ is defined by

$D_h\left(x,y\right):=h\left(x\right)-h\left(y\right)-\langle \nabla h\left(y\right),x-y\rangle .$

Since $h(\cdot )$ is convex, $D_h\left(x,y\right)\ge 0$, and $D_h\left(x,y\right)=0$ if only if when $x=y$.

Definition 2.11. We say that $\left(x^{*},y^{*},{\lambda }^{*}\right)$ is a critical point of the Augmented Lagrangian Function ${ℒ}_{\beta }^h(\cdot )$ (1.5) with Bregman distance if it satisfies

$\{\begin{array}{l}{A}^{\text{T}}{\lambda }^{\text{*}}\in \partial f\left(x^{\text{*}}\right),\\ {\lambda }^{\text{*}}=\nabla g\left(y^{\text{*}}\right),\\ A{x}^{\text{*}}+y^{\text{*}}=b.\end{array}$

Lemma 2.5. Let $\left\{w_k:=\left(x_k,y_k,{\lambda }_k\right)\right\}$ be the sequence generated by the symmetric Bregman ADMM (1.6). Then, we have

$\{\begin{array}{l}\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)=\frac{1}{\left(\alpha +1\right)\beta }\left({\lambda }_k-{\lambda }_{k+1}\right)+\frac{1}{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right),\\ \nabla g\left(y_{k+1}\right)={\lambda }_{k+1},\\ \nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)=\frac{1}{\left(\alpha +1\right)\beta }\left({\lambda }_k-{\lambda }_{k+1}\right)-\frac{\alpha }{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right).\end{array}$

Proof. Combining (1.6b) and (1.6d), we get

$\begin{array}{c}{\lambda }_{k+1}-{\lambda }_k=-\alpha \beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right)-\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)\\ =-\alpha \beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right)-\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right)\\ +\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right)-\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)\\ =-\left(\alpha +1\right)\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right)+\beta \left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right)\end{array}$

and thus

$\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)=\frac{1}{\left(\alpha +1\right)\beta }\left({\lambda }_k-{\lambda }_{k+1}\right)+\frac{1}{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right).$

Similarly, we combine (1.6b) and (1.6d) again

$\begin{array}{c}{\lambda }_{k+1}-{\lambda }_k=-\alpha \beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right)-\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)\\ =-\alpha \beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)-\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)\\ -\alpha \beta \left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right)\\ =-\left(\alpha +1\right)\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)-\alpha \beta \left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right),\end{array}$

thus, we obtain

$\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)=\frac{1}{\left(\alpha +1\right)\beta }\left({\lambda }_k-{\lambda }_{k+1}\right)-\frac{\alpha }{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right).$

From the optimality condition of (1.6c), we have

$0=\nabla g\left(y_{k+1}\right)-{\lambda }_{k+\frac{1}{2}}+\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right).$

Substituting (1.6d) into the above equation yields

$\nabla g\left(y_{k+1}\right)={\lambda }_{k+1}.$

This completes the proof.

3. Convergence Analysis

In this section, we analyze the convergence of the symmetric Bregman ADMM (1.6) and show that the sequence $\left\{w_k:=\left(x_k,y_k,{\lambda }_k\right)\right\}$ generated by the symmetric Bregman ADMM (1.6) converges to a critical point $\left\{w^{\text{*}}:=\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\right\}$ of ${ℒ}_h(\cdot )$ under the following assumptions.

Assumption A. Let $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper lower semicontinuous function, $g:{ℝ}^m\to ℝ$ be a continuously differentiable function with $\nabla g$ being D-Lipschitz continuous. Additionally, let $h\in \mathcal{G}\left(C\right)$ be a twice differentiable function on $C=int\left(dom\left(h\right)\right)$, which is 1-strong-convex, and whose $\nabla h$ is Lipschitz continuous with $L_h$ on any bounded subset of ${ℝ}^m$. We assume that

(i) $\alpha \in \left(-\frac{1}{2L_h^2+1},0\right]$, $\beta >\frac{2L{L}_h^2-2\alpha L{L}_h^3}{1+\alpha +2\alpha L_h^2}$, which implies

$\delta =\left(\frac{\beta }{L_h}-L+\frac{2\alpha L{L}_h+2\alpha \beta L_h+2\alpha L{L}_h^2}{\alpha +1}-\frac{2L^2{L}_h^2}{\left(\alpha +1\right)\beta }\right)>0.$

(ii) $\alpha \in \left[0,\frac{1}{2L_h^2-1}\right)$, $\beta >\frac{2\left(1+\alpha \right)L{L}_h^2+2\alpha L{L}_h^3}{1+\alpha -2\alpha L_h^2}$, which implies

$\delta =\left(\frac{\beta }{L_h}-L-\frac{2\alpha L{L}_h+2\alpha \beta L_h+2\alpha L{L}_h^2}{\alpha +1}-\frac{2L^2{L}_h^2}{\left(\alpha +1\right)\beta }\right)>0.$

(iii) $A^{\text{T}}A\underset{\_}{\succ }MI$ for some $M>0$.

Next, we examine the optimality conditions of (1.6). Invoking the optimality conditions, we have that

$\{\begin{array}{l}0\in \partial f\left(x_{k+1}\right)-A^{\text{T}}{\lambda }_k-\beta A^{\text{T}}{\left({\nabla }^{2}h\left(A{x}_{k+1}-b\right)\right)}^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right),\\ {\lambda }_{k+\frac{1}{2}}={\lambda }_k-\alpha \beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right),\\ 0=\nabla g\left(y_{k+1}\right)-{\lambda }_{k+\frac{1}{2}}+\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right),\\ {\lambda }_{k+1}={\lambda }_{k+\frac{1}{2}}-\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right).\end{array}$(3.1)

Lemma 3.1. Let $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ be the sequence generated by the symmetric Bregman ADMM (1.6), which is assumed to be bounded. Then we have

${ℒ}_{\beta }^h\left(w_{k+1}\right)\le {ℒ}_{\beta }^h\left(w_k\right)-\delta \cdot D_h\left(y_k,y_{k+1}\right).$(3.2)

Proof. From the definition of ${ℒ}_{\beta }^h$ in (1.5), it follows that

$\begin{array}{l}{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+\frac{1}{2}}\right)\\ =g\left(y_k\right)-\langle {\lambda }_{k+\frac{1}{2}},A{x}_{k+1}+y_k-b\rangle +\beta D_h\left(-y_k,A{x}_{k+1}-b\right)\\ -g\left(y_{k+1}\right)+\langle {\lambda }_{k+\frac{1}{2}},A{x}_{k+1}+y_{k+1}-b\rangle -\beta D_h\left(-y_{k+1},A{x}_{k+1}-b\right)\\ =g\left(y_k\right)-g\left(y_{k+1}\right)+\langle {\lambda }_{k+\frac{1}{2}},y_{k+1}-y_k\rangle +\beta D_h\left(-y_k,A{x}_{k+1}-b\right)\\ -\beta D_h\left(-y_{k+1},A{x}_{k+1}-b\right).\end{array}$(3.3)

Since the gradient of the function $g$ is D-Lipschitz continuous on $int\left(dom\left(h\right)\right)$, it follows that the function pair $\left(g,h\right)$ is L-smooth adaptable. Then, according to Lemma 2.1, we can obtain

$g\left(y_k\right)-g\left(y_{k+1}\right)\ge \langle \nabla g\left(y_{k+1}\right),y_k-y_{k+1}\rangle -L{D}_h\left(y_k,y_{k+1}\right).$(3.4)

By substituting inequality (3.4) into relation (3.3), we obtain

$\begin{array}{l}{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+\frac{1}{2}}\right)\\ \ge \langle \nabla g\left(y_{k+1}\right),y_k-y_{k+1}\rangle -L{D}_h\left(y_k,y_{k+1}\right)+\langle {\lambda }_{k+\frac{1}{2}},y_{k+1}-y_k\rangle \\ +\beta D_h\left(-y_k,A{x}_{k+1}-b\right)-\beta D_h\left(-y_{k+1},A{x}_{k+1}-b\right)\\ =\langle {\lambda }_{k+1}-{\lambda }_{k+\frac{1}{2}},y_k-y_{k+1}\rangle -L{D}_h\left(y_k,y_{k+1}\right)\end{array}$

$\begin{array}{l}+\beta D_h\left(-y_k,A{x}_{k+1}-b\right)-\beta D_h\left(-y_{k+1},A{x}_{k+1}-b\right)\\ =\langle {\lambda }_{k+1}-{\lambda }_{k+\frac{1}{2}},y_k-y_{k+1}\rangle -L{D}_h\left(y_k,y_{k+1}\right)+\beta D_h\left(-y_k,-y_{k+1}\right)\\ +\beta \left(D_h\left(-y_k,A{x}_{k+1}-b\right)-D_h\left(-y_k,-y_{k+1}\right)-D_h\left(-y_{k+1},A{x}_{k+1}-b\right)\right)\\ =\langle {\lambda }_{k+1}-{\lambda }_{k+\frac{1}{2}},y_k-y_{k+1}\rangle -L{D}_h\left(y_k,y_{k+1}\right)+\beta D_h\left(-y_k,-y_{k+1}\right)\\ +\beta \langle \nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right),y_k-y_{k+1}\rangle \\ =\langle {\lambda }_{k+1}-{\lambda }_{k+\frac{1}{2}},y_k-y_{k+1}\rangle -L{D}_h\left(y_k,y_{k+1}\right)+\beta D_h\left(-y_k,-y_{k+1}\right)\\ +\langle {\lambda }_{k+\frac{1}{2}}-{\lambda }_{k+1},y_k-y_{k+1}\rangle \\ =\beta D_h\left(-y_k,-y_{k+1}\right)-L{D}_h\left(y_k,y_{k+1}\right),\end{array}$

where the first and the third equalities follow from Lemma (2.5) and the three points identity (2.4) respectively.

Based on the 1-strong convexity and $L_h$-smoothness of the function $h(\cdot )$, we can obtain the following two inequalities respectively

$D_h\left(y_k,y_{k+1}\right)=h\left(y_k\right)-h\left(y_{k+1}\right)-\langle \nabla h\left(y_{k+1}\right),y_k-y_{k+1}\rangle \le \frac{L_h}{2}{\Vert y_k-y_{k+1}\Vert }^2,$

$D_h\left(-y_k,-y_{k+1}\right)=h\left(-y_k\right)-h\left(-y_{k+1}\right)-\langle \nabla h\left(-y_{k+1}\right),y_{k+1}-y_k\rangle \ge \frac{1}{2}{\Vert y_k-y_{k+1}\Vert }^2.$

Combining the above two inequalities, we get

$D_h\left(-y_k,-y_{k+1}\right)\ge \frac{1}{2}{\Vert y_{k+1}-y_k\Vert }^2\ge \frac{1}{L_h}\cdot D_h\left(y_k,y_{k+1}\right),$

thus, we can obtain

${ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+\frac{1}{2}}\right)\ge \left(\frac{\beta }{L_h}-L\right)D_h\left(y_k,y_{k+1}\right).$(3.5)

Next, by using (1.6b) and (1.5), we have

$\begin{array}{l}{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_k\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_{k+\frac{1}{2}}\right)+{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+1}\right)\\ =\langle {\lambda }_{k+\frac{1}{2}}-{\lambda }_k,A{x}_{k+1}+y_k-b\rangle +\langle {\lambda }_{k+1}-{\lambda }_{k+\frac{1}{2}},A{x}_{k+1}+y_{k+1}-b\rangle \\ =\langle -\alpha \beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right),A{x}_{k+1}+y_k-b\rangle \\ +\langle {\lambda }_{k+1}-{\lambda }_k+\alpha \beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right),A{x}_{k+1}+y_{k+1}-b\rangle \\ =\langle \alpha \beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right),y_{k+1}-y_k\rangle +\langle {\lambda }_{k+1}-{\lambda }_k,A{x}_{k+1}+y_{k+1}-b\rangle ,\end{array}$

where the second equality follows from (1.6b). Then, according to Lemma 2.5

$\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)=\frac{1}{\left(\alpha +1\right)\beta }\left({\lambda }_k-{\lambda }_{k+1}\right)-\frac{\alpha }{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right),$

$\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)=\frac{1}{\left(\alpha +1\right)\beta }\left({\lambda }_k-{\lambda }_{k+1}\right)+\frac{1}{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right),$

and, since $h(\cdot )$ is 1-strong-convex, we have

$\Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(A{x}_{k+1}-b\right)\Vert \ge \Vert A{x}_{k+1}+y_{k+1}-b\Vert .$

Now, we discuss the two cases based on the range of $\alpha$.

(i) $\alpha \in \left(-\frac{1}{2L_h^2+1},0\right]$, combining the above formulas

$\begin{array}{l}{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_k\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_{k+\frac{1}{2}}\right)+{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+1}\right)\\ \ge \langle \frac{\alpha \beta }{\left(\alpha +1\right)\beta }\left({\lambda }_k-{\lambda }_{k+1}\right)+\frac{\alpha \beta }{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right),y_{k+1}-y_k\rangle \\ -\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \Vert A{x}_{k+1}+y_{k+1}-b\Vert \\ \ge \langle \frac{\alpha }{\alpha +1}\left({\lambda }_k-{\lambda }_{k+1}\right)+\frac{\alpha \beta }{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right),y_{k+1}-y_k\rangle \\ -\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(A{x}_{k+1}-b\right)\Vert \\ \ge \langle \frac{\alpha }{\alpha +1}\left({\lambda }_k-{\lambda }_{k+1}\right)+\frac{\alpha \beta }{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right),y_{k+1}-y_k\rangle \\ -\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \left[\frac{1}{\left(\alpha +1\right)\beta }\Vert {\lambda }_{k+1}-{\lambda }_k\Vert -\frac{\alpha }{\alpha +1}\Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\Vert \right]\\ \ge \frac{\alpha }{\alpha +1}\Vert {\lambda }_k-{\lambda }_{k+1}\Vert \Vert y_{k+1}-y_k\Vert +\frac{\alpha \beta }{\alpha +1}\Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\Vert \Vert y_{k+1}-y_k\Vert \\ -\frac{1}{\left(\alpha +1\right)\beta }{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2+\frac{\alpha }{\alpha +1}\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\Vert .\end{array}$(3.6)

Subsequently, we claim that

$\Vert \nabla g\left(y_{k+1}\right)-\nabla g\left(y_k\right)\Vert \le L{L}_h\cdot \Vert y_{k+1}-y_k\Vert .$(3.7)

To prove (3.7), we consider two cases. When $y_k=y_{k+1}$, (3.7) holds obviously. Next, we assume $y_{k+1}\ne y_k$. Since $\nabla g(\cdot )$ is $D$-Lipschitz, we obtain

$\begin{array}{c}\Vert \nabla g\left(y_{k+1}\right)-\nabla g\left(y_k\right)\Vert \le L\cdot \frac{D_h\left(y_{k+1},y_k\right)+D_h\left(y_k,y_{k+1}\right)}{\Vert y_{k+1}-y_k\Vert }\\ =L\cdot \frac{\langle \nabla h\left(y_k\right)-\nabla h\left(y_{k+1}\right),y_k-y_{k+1}\rangle }{\Vert y_{k+1}-y_k\Vert }\\ \le L\cdot \frac{\Vert \nabla h\left(y_k\right)-\nabla h\left(y_{k+1}\right)\Vert \cdot \Vert y_k-y_{k+1}\Vert }{\Vert y_{k+1}-y_k\Vert }\\ \le L\cdot \frac{L_h{\Vert y_k-y_{k+1}\Vert }^2}{\Vert \Vert y_{k+1}-y_k\Vert \Vert }=L{L}_h\cdot \Vert y_{k+1}-y_k\Vert ,\end{array}$

where the second inequality is a consequence of $\nabla h$ is Lipschitz continuous with $L_h\left(L_h\ge 1\right)$ on any bounded subset of ${ℝ}^m$, that is

$\Vert \nabla h\left(y_{k+1}\right)-\nabla h\left(y_k\right)\Vert \le L_h\Vert y_{k+1}-y_k\Vert .$(3.8)

Since ${\lambda }_{k+1}=\nabla g\left(y_{k+1}\right)$, (3.7) becomes

$\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \le L{L}_h\Vert y_{k+1}-y_k\Vert .$(3.9)

Moreover, according to $h(\cdot )$ is 1-strong-convex, we get $D_h\left(y_k,y_{k+1}\right)\ge \frac{1}{2}{\Vert y_k-y_{k+1}\Vert }^2$, i.e.,

${\Vert y_k-y_{k+1}\Vert }^2\le 2D_h\left(y_k,y_{k+1}\right).$(3.10)

Substituting (3.8), (3.9) and (3.10) into (3.6), we conclude that

$\begin{array}{l}{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_k\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_{k+\frac{1}{2}}\right)+{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+1}\right)\\ \ge \frac{2\alpha L{L}_h}{\alpha +1}{D}_h\left(y_k,y_{k+1}\right)+\frac{2\alpha \beta L_h}{\alpha +1}{D}_h\left(y_k,y_{k+1}\right)\\ -\frac{2L^2{L}_h^2}{\left(\alpha +1\right)\beta }{D}_h\left(y_k,y_{k+1}\right)+\frac{2\alpha L{L}_h^2}{\alpha +1}{D}_h\left(y_k,y_{k+1}\right)\\ =\left(\frac{2\alpha L{L}_h+2\alpha \beta L_h+2\alpha L{L}_h^2}{\alpha +1}-\frac{2L^2{L}_h^2}{\left(\alpha +1\right)\beta }\right)D_h\left(y_k,y_{k+1}\right).\end{array}$ (3.11)

Then, combining (3.5) and (3.11), we obtain that

$\begin{array}{l}{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_k\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+1}\right)\\ \ge \left(\frac{\beta }{L_h}-L+\frac{2\alpha L{L}_h+2\alpha \beta L_h+2\alpha L{L}_h^2}{\alpha +1}-\frac{2L^2{L}_h^2}{\left(\alpha +1\right)\beta }\right)D_h\left(y_k,y_{k+1}\right).\end{array}$(3.12)

Consequently, according to (1.6a), we have

${ℒ}_{\beta }^h\left(x_k,y_k,{\lambda }_k\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_k\right)\ge 0.$(3.13)

Finally, summing the inequality (3.12) and (3.13), we conclude that

$\begin{array}{l}{ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{k+1}\right)\\ \ge \left(\frac{\beta }{L_h}-L+\frac{2\alpha L{L}_h+2\alpha \beta L_h+2\alpha L{L}_h^2}{\alpha +1}-\frac{2L^2{L}_h^2}{\left(\alpha +1\right)\beta }\right)D_h\left(y_k,y_{k+1}\right).\end{array}$

This completes the discussion for case (i), we now proceed to discuss case (ii).

(ii) $\alpha \in \left[0,\frac{1}{2L_h^2-1}\right)$, combining the same formulas

$\begin{array}{l}{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_k\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_{k+\frac{1}{2}}\right)+{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+1}\right)\\ \ge \langle \frac{\alpha \beta }{\left(\alpha +1\right)\beta }\left({\lambda }_k-{\lambda }_{k+1}\right)+\frac{\alpha \beta }{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right),y_{k+1}-y_k\rangle \\ -\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \Vert A{x}_{k+1}+y_{k+1}-b\Vert \\ \ge \langle \frac{\alpha }{\alpha +1}\left({\lambda }_k-{\lambda }_{k+1}\right)+\frac{\alpha \beta }{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right),y_{k+1}-y_k\rangle \end{array}$

$\begin{array}{l}-\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(A{x}_{k+1}-b\right)\Vert \\ \ge \langle \frac{\alpha }{\alpha +1}\left({\lambda }_k-{\lambda }_{k+1}\right)+\frac{\alpha \beta }{\alpha +1}\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right),y_{k+1}-y_k\rangle \\ -\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \left[\frac{1}{\left(\alpha +1\right)\beta }\Vert {\lambda }_{k+1}-{\lambda }_k\Vert +\frac{\alpha }{\alpha +1}\Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\Vert \right]\\ \ge -\frac{\alpha }{\alpha +1}\Vert {\lambda }_k-{\lambda }_{k+1}\Vert \Vert y_{k+1}-y_k\Vert -\frac{\alpha \beta }{\alpha +1}\Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\Vert \Vert y_{k+1}-y_k\Vert \\ -\frac{1}{\left(\alpha +1\right)\beta }{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2-\frac{\alpha }{\alpha +1}\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\Vert .\end{array}$(3.14)

Similarly, substituting (3.8), (3.9) and (3.10) into (3.14), we conclude that

$\begin{array}{l}{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_k\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_{k+\frac{1}{2}}\right)+{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+\frac{1}{2}}\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+1}\right)\\ \ge -\frac{2\alpha L{L}_h}{\alpha +1}{D}_h\left(y_k,y_{k+1}\right)-\frac{2\alpha \beta L_h}{\alpha +1}{D}_h\left(y_k,y_{k+1}\right)\\ -\frac{2L^2{L}_h^2}{\left(\alpha +1\right)\beta }{D}_h\left(y_k,y_{k+1}\right)-\frac{2\alpha L{L}_h^2}{\alpha +1}{D}_h\left(y_k,y_{k+1}\right)\\ =\left(-\frac{2\alpha L{L}_h+2\alpha \beta L_h+2\alpha L{L}_h^2}{\alpha +1}-\frac{2L^2{L}_h^2}{\left(\alpha +1\right)\beta }\right)D_h\left(y_k,y_{k+1}\right).\end{array}$ (3.15)

Then, combining (3.5) and (3.15), we obtain that

$\begin{array}{l}{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_k\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_{k+1},{\lambda }_{k+1}\right)\\ \ge \left(\frac{\beta }{L_h}-L-\frac{2\alpha L{L}_h+2\alpha \beta L_h+2\alpha L{L}_h^2}{\alpha +1}-\frac{2L^2{L}_h^2}{\left(\alpha +1\right)\beta }\right)D_h\left(y_k,y_{k+1}\right).\end{array}$(3.16)

Consequently, according to (1.6a), we have

${ℒ}_{\beta }^h\left(x_k,y_k,{\lambda }_k\right)-{ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_k\right)\ge 0.$(3.17)

Finally, summing the inequality (3.16) and (3.17), we conclude that

$\begin{array}{l}{ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{k+1}\right)\\ \ge \left(\frac{\beta }{L_h}-L-\frac{2\alpha L{L}_h+2\alpha \beta L_h+2\alpha L{L}_h^2}{\alpha +1}-\frac{2L^2{L}_h^2}{\left(\alpha +1\right)\beta }\right)D_h\left(y_k,y_{k+1}\right).\end{array}$

This completes the discussion for case (ii), and with this, the proof is complete.□

Remark 3.1. Since $\delta >0$, Lemma 3.1 implies that $\left\{{ℒ}_{\beta }^h\left(w_k\right)\right\}$ is monotonically nonincreasing. Note that when $\alpha =0$, we have $\beta >2L{L}_h^2$. This corresponds to the requirement in Tan and Guo [24] . Furthermore, when $h(\cdot )=\frac{1}{2}{\Vert \cdot \Vert }^2$, we have $\alpha =0$ and $L_h=1$, thus $\beta >2L$. This corresponds to the requirement in Guo et al. [19] .

Lemma 3.2. Let $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ be the sequence generated by the symmetric Bregman ADMM (1.6), which is assumed to be bounded. Then we have

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}{\Vert w_{k+1}-w_k\Vert }^2<+\infty .$

Proof: Considering that $\left\{w_k\right\}$ is bounded, it has at least one limit point. Let $w^{\text{*}}$ be a limit point of $\left\{w_k\right\}$ and let $\left\{w_{k_j}\right\}$ be the subsequence converging to it, i.e. $w_{k_j}\to w^{\text{*}}$. Given that ${ℒ}_{\beta }^h(\cdot )$ is a lower semicontinuous function, we can deduce that

${ℒ}_{\beta }^h\left(w^{*}\right)\le \underset{j\to +\infty }{\lim \inf }{ℒ}_{\beta }^h\left(w_{k_j}\right).$

Consequently, $\left\{{ℒ}_{\beta }^h\left(w_{k_j}\right)\right\}$ is bounded from below. Besides, the fact that $\left\{{ℒ}_{\beta }^h\left(w_k\right)\right\}$ is nonincreasing implies that $\left\{{ℒ}_{\beta }^h\left(w_{k_j}\right)\right\}$ is convergent. Moreover, $\left\{{ℒ}_{\beta }^h\left(w_k\right)\right\}$ is convergent, and ${ℒ}_{\beta }^h\left(w_k\right)\ge {ℒ}_{\beta }^h\left(w^{\text{*}}\right)$. Based on the Equation (3.2), we can obtain

$\delta \cdot D_h\left(y_k,y_{k+1}\right)\le {ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w_{k+1}\right).$

Summing up the above inequality from $k=0$ to $n$, we conclude that

$\underset{k=0}{\overset{n}{{\displaystyle \sum }}}\delta D_h\left(y_k,y_{k+1}\right)\le {ℒ}_{\beta }^h\left(w_0\right)-{ℒ}_{\beta }^h\left(w_{n+1}\right)\le {ℒ}_{\beta }^h\left(w_0\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)<+\infty .$

Owing to $\delta >0$, we have $\underset{k=0}{\overset{\infty }{{\displaystyle \sum }}}{D}_h\left(y_k,y_{k+1}\right)<+\infty$, which implies $\underset{k=0}{\overset{\infty }{{\displaystyle \sum }}}{\Vert y_{k+1}-y_k\Vert }^2<+\infty$. Consequently, according to (3.9), we get that

$\underset{k=0}{\overset{\infty }{{\displaystyle \sum }}}{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2<+\infty .$

Recall Lemma 2.5, we have

${\lambda }_{k+1}={\lambda }_k-\beta \left[\left(\alpha +1\right)\left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_k\right)\right)-\left(\nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\right)\right],$

${\lambda }_k={\lambda }_{k-1}-\beta \left[\left(\alpha +1\right)\left(\nabla h\left(A{x}_k-b\right)-\nabla h\left(-y_{k-1}\right)\right)-\left(\nabla h\left(-y_k\right)-\nabla h\left(-y_{k-1}\right)\right)\right].$

Combining the two equalities, we obtain

$\begin{array}{l}\left(\alpha +1\right)\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(A{x}_k-b\right)\right)\\ =\left({\lambda }_k-{\lambda }_{k-1}\right)-\left({\lambda }_{k+1}-{\lambda }_k\right)-\alpha \beta \left(\nabla h\left(-y_{k-1}\right)-\nabla h\left(-y_k\right)\right)\\ -\beta \left(\nabla h\left(-y_k\right)-\nabla h\left(-y_{k+1}\right)\right).\end{array}$

Then, we use the 1-strong-convex of $h(\cdot )$ and Assumption A(iii), we can follow

$M{\Vert x_{k+1}-x_k\Vert }^2\le {\Vert A{x}_{k+1}-A{x}_k\Vert }^2\le {\Vert \nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(A{x}_k-b\right)\Vert }^2.$

Thus, combining the above two formulas, we obtain that

$\begin{array}{l}{\left(\alpha +1\right)}^2{\beta }^{2}M{\Vert x_{k+1}-x_k\Vert }^2\\ \le {\left(\alpha +1\right)}^2{\beta }^2{\Vert A{x}_{k+1}-A{x}_k\Vert }^2\le {\left(\alpha +1\right)}^2{\beta }^2{\Vert \nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(A{x}_k-b\right)\Vert }^2\\ \le 4({\Vert {\lambda }_k-{\lambda }_{k-1}\Vert }^2+{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2+{\alpha }^2{\beta }^2{\Vert \nabla h\left(-y_{k-1}\right)-\nabla h\left(-y_k\right)\Vert }^2\\ +{\beta }^2{\Vert \nabla h\left(-y_k\right)-\nabla h\left(-y_{k+1}\right)\Vert }^2)\\ \le 4\left({\Vert {\lambda }_k-{\lambda }_{k-1}\Vert }^2+{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2+{\alpha }^2{\beta }^2{L}_h^2{\Vert y_k-y_{k-1}\Vert }^2+{\beta }^2{L}_h^2{\Vert y_{k+1}-y_k\Vert }^2\right),\end{array}$(3.18)

where $M>0$. Then, the last inequality implies $\underset{k=0}{\overset{\infty }{{\displaystyle \sum }}}{\Vert x_{k+1}-x_k\Vert }^2<+\infty$.

Thus, $\underset{k=0}{\overset{\infty }{{\displaystyle \sum }}}{\Vert w_{k+1}-w_k\Vert }^2<+\infty$. This completes the proof.□

Lemma 3.3. Let $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ be the sequence generated by the symmetric Bregman ADMM (1.6), which is supposed to be bounded, and given that Assumption A holds. For any positive integer $k$, we define

$\{\begin{array}{l}{x}_{k+1}^{\text{*}}:=A^{\text{T}}{\lambda }_k-A^{\text{T}}{\lambda }_{k+1}+\beta A^{\text{T}}\langle \nabla h^2\left(A{x}_{k+1}-b\right),y_{k+1}-y_k\rangle ,\\ y_{k+1}^{\text{*}}:={\lambda }_{k+\frac{1}{2}}-{\lambda }_{k+1},\\ {\lambda }_{k+1}^{\text{*}}:=-A{x}_{k+1}-y_{k+1}+b.\end{array}$

Then, $\left(x_{k+1}^{\text{*}},y_{k+1}^{\text{*}},{\lambda }_{k+1}^{\text{*}}\right)\in \partial {ℒ}_{\beta }^h\left(w_{k+1}\right)$ and there exists $\xi >0$ such that

$d\left(0,\partial {ℒ}_{\beta }^h\left(w_{k+1}\right)\right)\le \xi \Vert y_{k+1}-y_k\Vert .$

Proof: By definition of function ${ℒ}_{\beta }^h(\cdot )$, we can obtain that

$\{\begin{array}{l}{\partial }_x{ℒ}_{\beta }^h\left(w_{k+1}\right)=\partial f\left(x_{k+1}\right)-A^{\text{T}}{\lambda }_{k+1}-\beta A^{\text{T}}\langle {\nabla }^{2}h\left(A{x}_{k+1}-b\right),-A{x}_{k+1}-y_{k+1}+b\rangle ,\\ {\partial }_y{ℒ}_{\beta }^h\left(w_{k+1}\right)=\nabla g\left(y_{k+1}\right)-{\lambda }_{k+1}+\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right),\\ {\partial }_{\lambda }{ℒ}_{\beta }^h\left(w_{k+1}\right)=-\left(A{x}_{k+1}+y_{k+1}-b\right).\end{array}$(3.19)

Combining the optimality condition (3.1) with equation (3.19), we get

$\{\begin{array}{l}{A}^{\text{T}}{\lambda }_k-A^{\text{T}}{\lambda }_{k+1}+\beta A^{\text{T}}\langle \nabla h^2\left(A{x}_{k+1}-b\right),y_{k+1}-y_k\rangle \in {\partial }_x{ℒ}_{\beta }^h\left(w_{k+1}\right),\\ {\lambda }_{k+\frac{1}{2}}-{\lambda }_{k+1}\in {\partial }_y{ℒ}_{\beta }^h\left(w_{k+1}\right),\\ -A{x}_{k+1}-y_{k+1}+b\in {\partial }_{\lambda }{ℒ}_{\beta }^h\left(w_{k+1}\right).\end{array}$

Then, according to Lemma 2.2, we obtain that $\left(x_{k+1}^{\text{*}},y_{k+1}^{\text{*}},{\lambda }_{k+1}^{\text{*}}\right)\in \partial {ℒ}_{\beta }^h\left(w_{k+1}\right)$.

Furthermore,

$\langle \nabla h^2\left(A{x}_{k+1}-b\right),y_{k+1}-y_k\rangle \le \Vert \nabla h^2\left(A{x}_{k+1}-b\right)\Vert \Vert y_{k+1}-y_k\Vert \le L_h\Vert y_{k+1}-y_k\Vert .$

In addition,

$\begin{array}{c}\Vert A{x}_{k+1}+y_{k+1}-b\Vert \le \Vert \nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\Vert \\ \le \frac{1}{\left(\alpha +1\right)\beta }\Vert {\lambda }_k-{\lambda }_{k+1}\Vert +\frac{\alpha }{\alpha +1}\Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\Vert \\ \le \frac{1}{\left(\alpha +1\right)\beta }\Vert {\lambda }_k-{\lambda }_{k+1}\Vert +\frac{\alpha L_h}{\alpha +1}\Vert y_k-y_{k+1}\Vert .\end{array}$

On the other hand,

$\begin{array}{c}\Vert {\lambda }_{k+\frac{1}{2}}-{\lambda }_{k+1}\Vert =\Vert \beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)\Vert \\ \le \frac{1}{\alpha +1}\Vert {\lambda }_k-{\lambda }_{k+1}\Vert +\frac{\alpha \beta }{\alpha +1}\Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(-y_k\right)\Vert \\ \le \frac{1}{\alpha +1}\Vert {\lambda }_k-{\lambda }_{k+1}\Vert +\frac{\alpha \beta L_h}{\alpha +1}\Vert y_{k+1}-y_k\Vert .\end{array}$

Based on the above relationship, we know that there exist ${\xi }_1,{\xi }_2>0$ such that

$\Vert \left(x_{k+1}^{\text{*}},y_{k+1}^{\text{*}},{\lambda }_{k+1}^{\text{*}}\right)\Vert \le {\xi }_1\Vert y_{k+1}-y_k\Vert +{\xi }_2\Vert {\lambda }_{k+1}-{\lambda }_k\Vert .$

Defining $\xi :={\xi }_1+{\xi }_{2}L{L}_h$ and using (3.9), we obtain

$d\left(0,\partial {ℒ}_{\beta }^h\left(w_{k+1}\right)\right)\le \Vert \left(x_{k+1}^{\text{*}},y_{k+1}^{\text{*}},{\lambda }_{k+1}^{\text{*}}\right)\Vert \le \xi \Vert y_{k+1}-y_k\Vert .$

This completes the proof.

Lemma 3.4. Let $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ be the sequence generated by the symmetric Bregman ADMM (1.6), which is supposed to be bounded. Let $S\left(w_0\right)$ denote the set of its limit points. Then

(i) $S\left(w_0\right)$ is a nonempty compact set, and

$d\left(w_k,S\left(w_0\right)\right)\to 0$, as $k\to +\infty$;

(ii) $S\left(w_0\right)\subset crit{ℒ}_{\beta }^h$, where $crit{ℒ}_{\beta }^h$ denotes the set of all stationary points of ${ℒ}_{\beta }^h$;

(iii) ${ℒ}_{\beta }^h(\cdot )$ is finite and constant on $S\left(w_0\right)$, which equals to

$\underset{k\in \mathbb{N}}{\text{inf}}{ℒ}_{\beta }^h\left(w_k\right)=\underset{k\to +\infty }{\text{lim}}{ℒ}_{\beta }^h\left(w_k\right).$

Proof:

(i) The proposition is immediately derived from the definition of limit points.

(ii) We assume that $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\in S\left(w_0\right)$, then there exists a subsequence $\left\{\left(x_{k_j},y_{k_j},{\lambda }_{k_j}\right)\right\}$ that converges to $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$. Note that Lemma 3.2, we have

$\Vert w_{k+1}-w_k\Vert \to 0.$(3.20)

Consequently, we deduce that $\left\{\left(x_{k_j+1},y_{k_j+1},{\lambda }_{k_j+1}\right)\right\}$ also converges to $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$. Given that $x_{k+1}$ is the minimizer of ${ℒ}_{\beta }^h\left(x,y_k,{\lambda }_k\right)$ concerning the variable $x$, we have

${ℒ}_{\beta }^h\left(x_{k+1},y_k,{\lambda }_k\right)\le {ℒ}_{\beta }^h\left(x^{\text{*}},y_k,{\lambda }_k\right).$(3.21)

On one hand, according to (3.20), (3.21) and the continuity of ${ℒ}_{\beta }^h(\cdot )$ with respect to $y$ and $\lambda$, we get

$\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^h\left(x_{k_j+1},y_{k_j},{\lambda }_{k_j}\right)=\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^h\left(x_{k_j+1},y_{k_j+1},{\lambda }_{k_j+1}\right)\le {ℒ}_{\beta }^h\left(x^{*},y^{*},{\lambda }^{*}\right).$(3.22)

On the other hand, using the lower semicontinuity of ${ℒ}_{\beta }^h(\cdot )$, we have

$\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^h\left(x_{k_j+1},y_{k_j+1},{\lambda }_{k_j+1}\right)\ge {ℒ}_{\beta }^h\left(x^{*},y^{*},{\lambda }^{*}\right).$(3.23)

The above two relations (3.22) and (3.23) imply that

$\underset{j\to +\infty }{\text{lim}}f\left(x_{k_j+1}\right)=f\left(x^{\text{*}}\right).$

Taking the limit in the optimality conditions (3.1) along the subsequence $\left\{\left(x_{k_j+1},y_{k_j+1},{\lambda }_{k_j+1}\right)\right\}$, and utilizing (3.20) yields

$\{\begin{array}{l}{A}^{\text{T}}{\lambda }^{\text{*}}\in \partial f\left(x^{\text{*}}\right),\\ {\lambda }^{\text{*}}=\nabla g\left(y^{\text{*}}\right),\\ \nabla h\left(A{x}^{\text{*}}-b\right)-\nabla h\left(-y^{\text{*}}\right)=0.\end{array}$

Thus, $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$ is a critical point of (5), which implies that $w^{\text{*}}\in crit{ℒ}_{\beta }^h$.

(iii) For any point $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\in S\left(w_0\right)$, there exists a subsequence $\left\{\left(x_{k_j},y_{k_j},{\lambda }_{k_j}\right)\right\}$ that converges to $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$ as $j\to +\infty$. By merging equations (22) and (23) with the observation that the sequence $\left\{{ℒ}_{\beta }^h\left(w_k\right)\right\}$ is nonincreasing, we have

$\underset{k\to +\infty }{\text{lim}}{ℒ}_{\beta }^h\left(x_k,y_k,{\lambda }_k\right)={ℒ}_{\beta }^h\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right).$

Therefore, ${ℒ}_{\beta }^h(\cdot )$ is constant on $S\left(w_0\right)$. Furthermore, we also have $\underset{k\in \mathbb{N}}{\text{inf}}{ℒ}_{\beta }^h\left(w_k\right)=\underset{k\to +\infty }{\text{lim}}{ℒ}_{\beta }^h\left(w_k\right)$.

Hence, we have complete the proof.

Now we present the main convergence result for the symmetric Bregman ADMM (1).

Theorem 3.1. Let $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ be the sequence generated by the symmetric Bregman ADMM (1.6), which is assumed to be bounded. Suppose that ${ℒ}_{\beta }^h(\cdot )$ is a KL function, then $\left\{w_k\right\}$ has finite length, that is

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}\Vert w_{k+1}-w_k\Vert <+\infty ,$

and as a consequence, $\left\{w_k\right\}$ converges to a critical point of ${ℒ}_{\beta }^h(\cdot )$.

Proof: Based on the proof of Lemma 3.4, it implies that ${ℒ}_{\beta }^h\left(w_k\right)\to {ℒ}_{\beta }^h\left(w^{\text{*}}\right)$ for all $w^{\text{*}}\in S\left(w_0\right)$. Next, we will consider two cases.

(i) If there exists an integer $k_0$ for which ${ℒ}_{\beta }^h\left(w_{k_0}\right)={ℒ}_{\beta }^h\left(w^{\text{*}}\right)$. Then according to Remark 3.1 and (2), we have

$\delta D_h\left(y_k,y_{k+1}\right)\le {ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w_{k+1}\right)\le {ℒ}_{\beta }^h\left(w_{k_0}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)=0,$

for any $k>k_0$. Consequently, we conclude that $y_{k+1}=y_k$ for any $k>k_0$. Combining (9) and (18), we further deduce that ${\lambda }_{k+1}={\lambda }_k$ and $x_{k+1}=x_k$ for any $k>k_0+1$, which implies that $w_{k+1}=w_k$. As a result, the assertion is substantiated.

(ii) If ${ℒ}_{\beta }^h\left(w_k\right)>{ℒ}_{\beta }^h\left(w^{\text{*}}\right)$ for all $k$. Considering that $d\left(w_k,S\left(w_0\right)\right)\to 0$, there exists $k_1>0$, such that $d\left(w_k,S\left(w_0\right)\right)<\epsilon$ $\left(\forall \epsilon >0\right)$ for any $k>k_1$. Furthermore, with ${ℒ}_{\beta }^h\left(w_k\right)\to {ℒ}_{\beta }^h\left(w^{\text{*}}\right)$, it follows that there exists $k_2>0$, such that ${ℒ}_{\beta }^h\left(w_k\right)<{ℒ}_{\beta }^h\left(w^{\text{*}}\right)+\eta$ $\left(\forall \eta >0\right)$ for any $k>k_2$.

Thus, when $k>\tilde{k}=\text{max}\left\{k_1,k_2\right\}$ for all $\epsilon ,\eta >0$, we derive the following conclusions

$d\left(w_k,S\left(w_0\right)\right)<\epsilon ,{ℒ}_{\beta }^h\left(w^{\text{*}}\right)<{ℒ}_{\beta }^h\left(w_k\right)<{ℒ}_{\beta }^h\left(w^{\text{*}}\right)+\eta .$

Since ${ℒ}_{\beta }^h(\cdot )$ is constant on $S\left(w_0\right)$ and $S\left(w_0\right)$ is a nonempty compact set, we use Lemma 2.3 with $\Omega =S\left(w_0\right)$ to derive that for any $k>\tilde{k}$,

${\phi }^{\prime }\left({ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)d\left(0,\partial {ℒ}_{\beta }^h\left(w_k\right)\right)\ge 1.$(3.24)

Relying on the fact that ${ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w_{k+1}\right)={ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)-\left({ℒ}_{\beta }^h\left(w_{k+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)$, and the concavity of $\phi (\cdot )$, it follows that

$\begin{array}{l}\phi \left({ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^h\left(w_{k+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)\\ \ge {\phi }^{\prime }\left({ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)\left({ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w_{k+1}\right)\right).\end{array}$

Combining $d\left(0,\partial {ℒ}_{\beta }^h\left(w_k\right)\right)\le \xi \Vert y_k-y_{k-1}\Vert$ with the above inequality, ${\phi }^{\prime }\left({ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)>0$ and inequality (3.24), we can obtain

$\begin{array}{l}{ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w_{k+1}\right)\le \frac{\phi \left({ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^h\left(w_{k+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)}{{\phi }^{\prime }\left({ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)}\\ \le d\left(0,\partial {ℒ}_{\beta }^h\left(w_k\right)\right)\left[\phi \left({ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^h\left(w_{k+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)\right]\\ \le \xi \Vert y_k-y_{k-1}\Vert \left[\phi \left({ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^h\left(w_{k+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)\right].\end{array}$(3.25)

For convenience, for all $p,q\in \mathbb{N}$, we define ${\text{Δ}}_{p,q}:=\phi \left({ℒ}_{\beta }^h\left(w_p\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^h\left(w_q\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)$.

Hence, (3.25) can be simplified as

${ℒ}_{\beta }^h\left(w_k\right)-{ℒ}_{\beta }^h\left(w_{k+1}\right)\le \xi \Vert y_k-y_{k-1}\Vert {\text{Δ}}_{k,k+1}.$(3.26)

Combining inequality (3.26) with Lemma 2, we get that for all $k>\tilde{k}$,

$\frac{\delta }{2}{\Vert y_k-y_{k+1}\Vert }^2\le \delta D_h\left(y_k,y_{k+1}\right)\le \xi \Vert y_k-y_{k-1}\Vert {\text{Δ}}_{k,k+1}.$

Then,

$\Vert y_k-y_{k+1}\Vert \le \sqrt{\frac{2\xi }{\delta }{\text{Δ}}_{k,k+1}}{\Vert y_k-y_{k-1}\Vert }^{1/2}.$

Applying the fact that $2\sqrt{\alpha \beta }\le \alpha +\beta$, we obtain

$2\Vert y_k-y_{k+1}\Vert \le \Vert y_k-y_{k-1}\Vert +\frac{2\xi }{\delta }{\text{Δ}}_{k,k+1}.$(3.27)

Summing up (3.27) over for $k=\tilde{k}+1,\cdots ,m$, we have

$2\underset{k=\tilde{k}+1}{\overset{m}{{\displaystyle \sum }}}\Vert y_{k+1}-y_k\Vert \le \underset{k=\tilde{k}+1}{\overset{m}{{\displaystyle \sum }}}\Vert y_k-y_{k-1}\Vert +\frac{2\xi }{\delta }{\text{Δ}}_{\tilde{k}+1,m+1}.$

Note that $\phi \left({ℒ}_{\beta }^h\left(w_{m+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)>0$ from Definition 2.8. Taking $m\to +\infty$, we have

$\underset{k=\tilde{k}+1}{\overset{+\infty }{{\displaystyle \sum }}}\Vert y_{k+1}-y_k\Vert \le \Vert y_{\tilde{k}+1}-y_{\tilde{k}}\Vert +\frac{2\xi }{\delta }\phi \left({ℒ}_{\beta }^h\left(w_{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right).$(3.28)

Therefore,

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}\Vert y_{k+1}-y_k\Vert <+\infty .$(3.29)

Combining (3.9) and (3.29), we get

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}\Vert {\lambda }_{k+1}-{\lambda }_k\Vert <+\infty .$(3.30)

Using (3.18), we obtain

$\begin{array}{c}\Vert x_{k+1}-x_k\Vert \le \frac{2}{\left(\alpha +1\right)\beta \sqrt{M}}({\Vert {\lambda }_k-{\lambda }_{k-1}\Vert }^2+{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2\\ +{\alpha }^2{\beta }^2{L}_h^2{\Vert y_k-y_{k-1}\Vert }^2+{{\beta }^2{L}_h^2{\Vert y_{k+1}-y_k\Vert }^2)}^{\frac{1}{2}}\\ \le \frac{2}{\left(\alpha +1\right)\beta \sqrt{M}}(\Vert {\lambda }_k-{\lambda }_{k-1}\Vert +\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \\ +\alpha \beta L_h\Vert y_k-y_{k-1}\Vert +\beta L_h\Vert y_{k+1}-y_k\Vert ).\end{array}$

Combining above inequality with (3.29) and (3.30), we have

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}\Vert x_{k+1}-x_k\Vert <+\infty .$(3.31)

Furthermore, we note that

$\begin{array}{c}\Vert w_{k+1}-w_k\Vert ={\left({\Vert x_{k+1}-x_k\Vert }^2+{\Vert y_{k+1}-y_k\Vert }^2+{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2\right)}^{1/2}\\ \le \Vert x_{k+1}-x_k\Vert +\Vert y_{k+1}-y_k\Vert +\Vert {\lambda }_{k+1}-{\lambda }_k\Vert .\end{array}$

Using (3.29), (3.30) and (3.31), it follows that

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}\Vert w_{k+1}-w_k\Vert <+\infty ,$

which implies that $\left\{w_k\right\}$ is a Cauchy sequence and thus convergent. The assertion follows from Lemma 3.4 immediately.□

Next, we provide the essential sufficient conditions to establish that the sequence $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ generated by the symmetric Bregman ADMM (1.6) is bounded.

Lemma 3.5. Let $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ be the sequence generated by the symmetric Bregman ADMM (1.6). Suppose that

$\underset{y}{\text{inf}}\left\{g\left(y\right)-\frac{1}{2L}{\Vert \nabla g\left(y\right)\Vert }^2\right\}=:\bar{g}>-\infty .$

If at least one of the following statements is true:

(i) $\underset{\Vert x\Vert \to +\infty }{\text{liminf}}f\left(x\right)=+\infty$.

(ii) $\underset{x}{\text{inf}}f\left(x\right)>-\infty$ and $\underset{\Vert y\Vert \to +\infty }{\text{liminf}}g\left(y\right)=+\infty$.

Then, the sequence $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ is bounded.

Proof. Suppose that condition (i) holds. From Lemma 3.1, we have

${ℒ}_{\beta }^h\left(x_k,y_k,{\lambda }_k\right)\le {ℒ}_{\beta }^h\left(x_1,y_1,{\lambda }_1\right).$

which implies

${ℒ}_{\beta }^h\left(x_1,y_1,{\lambda }_1\right)\ge f\left(x_k\right)+g\left(y_k\right)-\langle {\lambda }_k,A{x}_k+y_k-b\rangle +\beta D_h\left(-y_k,A{x}_k-b\right).$(3.32)

Using the 1-strong convexity of the function $h(\cdot )$ and the definition of $D_h\left(\cdot ,\cdot \right)$, we have

$\begin{array}{c}{D}_h\left(-y_k,A{x}_k-b\right)=h\left(-y_k\right)-h\left(A{x}_k-b\right)-\langle \nabla h\left(A{x}_k-b\right),-y_k-A{x}_k-b\rangle \\ \ge \frac{1}{2}{\Vert A{x}_k+y_k-b\Vert }^2.\end{array}$ (3.33)

Combining (3.32) with (3.33) and using the fact ${\lambda }_k=\nabla g\left(y_k\right)$, we get

$\begin{array}{c}{ℒ}_{\beta }^h\left(x_1,y_1,{\lambda }_1\right)\ge f\left(x_k\right)+g\left(y_k\right)-\langle {\lambda }_k,A{x}_k+y_k-b\rangle +\frac{\beta }{2}{\Vert A{x}_k+y_k-b\Vert }^2\\ =f\left(x_k\right)+g\left(y_k\right)-\frac{1}{2\beta }{\Vert {\lambda }_k\Vert }^2+\frac{\beta }{2}{\Vert A{x}_k+y_k-b-\frac{1}{\beta }{\lambda }_k\Vert }^2\\ =f\left(x_k\right)+\left(g\left(y_k\right)-\frac{1}{2L}{\Vert \nabla g\left(y_k\right)\Vert }^2\right)+\left(\frac{1}{2L}-\frac{1}{2\beta }\right){\Vert {\lambda }_k\Vert }^2\\ +\frac{\beta }{2}{\Vert A{x}_k+y_k-b-\frac{1}{\beta }{\lambda }_k\Vert }^2\\ \ge f\left(x_k\right)+\bar{g}+\left(\frac{1}{2L}-\frac{1}{2\beta }\right){\Vert {\lambda }_k\Vert }^2+\frac{\beta }{2}{\Vert A{x}_k+y_k-b-\frac{1}{\beta }{\lambda }_k\Vert }^2.\end{array}$(34)

Note that (i) implies that $\underset{x}{\text{inf}}f\left(x\right)>-\infty$. When $\alpha \in \left(-\frac{1}{2L_h^2+1},0\right]$, we have $\beta >\frac{2L{L}_h^2-2\alpha L{L}_h^3}{1+\alpha +2\alpha L_h^2}>2L$, besides, when $\alpha \in \left[0,\frac{1}{2L_h^2-1}\right)$, we have $\beta >\frac{2\left(1+\alpha \right)L{L}_h^2+2\alpha L{L}_h^3}{1+\alpha -2\alpha L_h^2}>2L$. Therefore, we can derive that $\left\{x_k\right\}$, $\left\{{\lambda }_k\right\}$ and $\left\{\frac{\beta }{2}{\Vert A{x}_k+y_k-b-\frac{1}{\beta }{\lambda }_k\Vert }^2\right\}$ are bounded. Consequently, $\left\{y_k\right\}$ is also bounded, and hence $\left\{w_k\right\}$ is bounded.

Next, suppose condition (ii) holds. Using ${\lambda }_k\in \partial g\left(y_k\right)$ and (3.34), we get that

$\begin{array}{c}{ℒ}_{\beta }^h\left(x_1,y_1,{\lambda }_1\right)\ge f\left(x_k\right)+g\left(y_k\right)-\frac{1}{2\beta }{\Vert {\lambda }_k\Vert }^2+\frac{\beta }{2}{\Vert A{x}_k+y_k-b-\frac{1}{\beta }{\lambda }_k\Vert }^2\\ =f\left(x_k\right)+\frac{1}{2}g\left(y_k\right)+\left(\frac{1}{2}g\left(y_k\right)-\frac{1}{2L}{\Vert \nabla g\left(y_k\right)\Vert }^2\right)\\ +\left(\frac{1}{4L}-\frac{1}{2\beta }\right){\Vert \lambda \Vert }^2+\frac{\beta }{2}{\Vert A{x}_k+y_k-b-\frac{1}{\beta }{\lambda }_k\Vert }^2\\ \ge f\left(x_k\right)+\frac{1}{2}g\left(y_k\right)+\frac{1}{2}\bar{g}+\left(\frac{1}{4L}-\frac{1}{2\beta }\right){\Vert {\lambda }_k\Vert }^2\\ +\frac{\beta }{2}{\Vert A{x}_k+y_k-b-\frac{1}{\beta }{\lambda }_k\Vert }^2.\end{array}$

Note that $\underset{\Vert y\Vert \to \infty }{\lim \inf }g\left(y\right)=+\infty$ implies that $\underset{y}{\text{inf}}g\left(y\right)>-\infty$. When $\alpha \in \left(-\frac{1}{2L_h^2+1},0\right]$, we have $\beta >\frac{2L{L}_h^2-2\alpha L{L}_h^3}{1+\alpha +2\alpha L_h^2}>2L$, besides, when $\alpha \in \left[0,\frac{1}{2L_h^2-1}\right)$, we have $\beta >\frac{2\left(1+\alpha \right)L{L}_h^2+2\alpha L{L}_h^3}{1+\alpha -2\alpha L_h^2}>2L$. Therefore, we conclude that the sequences $\left\{y_k\right\}$, $\left\{{\lambda }_k\right\}$ and $\left\{\frac{\beta }{2}{\Vert A{x}_k+y_k-b-\frac{1}{\beta }{\lambda }_k\Vert }^2\right\}$ are bounded. Then, from (3.18), it follows that $\left\{x_k\right\}$ is also bounded, and as a consequence, $\left\{w_k\right\}$ is bounded. This completes the proof.

Theorem 3.2. (Convergence rate) Let $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ be the sequence generated by the symmetric Bregman ADMM (1.6) and converge to $\left\{w^{*}=\left(x^{*},y^{*},{\lambda }^{*}\right)\right\}$. Assuming that ${ℒ}_{\beta }^h(\cdot )$ has the KL property at $\left(x^{*},y^{*},{\lambda }^{*}\right)$ with $\phi \left(s\right)=c{s}^{1-\theta }$, $\theta \in \left[0,1\right)$, $c>0$. Then, the following results hold

(i) If $\theta =0$, then the sequence $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ converges in a finite number of steps.

(ii) If $\theta \in \left(0,\frac{1}{2}\right]$, then there exists $c_1>0$ and $\tau \in \left[0,1\right)$ such that

$\Vert \left(x_k,y_k,{\lambda }_k\right)-\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\Vert \le c_1{\tau }^k.$

(iii) If $\theta \in \left(\frac{1}{2},1\right)$, then there exists $c_2>0$ such that

$\Vert \left(x_k,y_k,{\lambda }_k\right)-\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\Vert \le c_2{k}^{\left(\theta -1\right)/\left(2\theta -1\right)}.$

Proof: Firstly, consider the case that $\theta =0$, we have $\phi \left(s\right)=cs$ and ${\phi }^{\prime }\left(s\right)=c$. Proof by contradiction, suppose that $\left\{w_k=\left(x_k,y_k,{\lambda }_k\right)\right\}$ does not converge in a finite number of steps, and then, the KL property at $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$ yields $c\cdot d\left(0,\partial {ℒ}_{\beta }^h\left(w_k\right)\right)\ge 1$ for any sufficiently large $k$, which is contrary to Lemma 3.1.

Secondly, consider that $\theta >0$ and set ${\text{Δ}}_k=\underset{i=k}{\overset{+\infty }{{\displaystyle \sum }}}\Vert y_{i+1}-y_i\Vert$ for $k\ge 0$. By the triangle inequality, we derive that ${\text{Δ}}_k\ge \Vert y_k-y^{\text{*}}\Vert$, and hence it is able to estimate ${\text{Δ}}_k$. With these notations, it follows from (3.28) that

${\text{Δ}}_{\tilde{k}+1}\le {\text{Δ}}_{\tilde{k}}-{\text{Δ}}_{\tilde{k}+1}+\frac{2\xi }{\delta }\phi \left({ℒ}_{\beta }^h\left(w_{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right).$

Invoking the KL property of ${ℒ}_{\beta }^h(\cdot )$ at $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$, we conclude that

${\phi }^{\prime }\left({ℒ}_{\beta }^h\left(w_{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)d\left(0,\partial {ℒ}_{\beta }^h\left(w_{\tilde{k}+1}\right)\right)\ge 1.$

This can be taken to imply

${\left({ℒ}_{\beta }^h\left(w_{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)}^{\theta }\le c\cdot \left(1-\theta \right)d\left(0,\partial {ℒ}_{\beta }^h\left(w_{\tilde{k}+1}\right)\right).$(3.35)

According to Lemma 3.1, we get

$d\left(0,\partial {ℒ}_{\beta }^h\left(w_{\tilde{k}+1}\right)\right)\le \xi \Vert y_{\tilde{k}+1}-y_{\tilde{k}}\Vert =\xi \left({\text{Δ}}_{\tilde{k}}-{\text{Δ}}_{\tilde{k}+1}\right).$(3.36)

Combining (3.35) and (3.36), it follows that there exists $\gamma >0$ such that

$\phi \left({ℒ}_{\beta }^h\left(w_{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)=c\cdot {\left({ℒ}_{\beta }^h\left(w_{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)}^{1-\theta }\le \gamma {\left({\text{Δ}}_{\tilde{k}}-{\text{Δ}}_{\tilde{k}+1}\right)}^{\left(1-\theta \right)/\theta }.$

Therefore,

${\text{Δ}}_{\tilde{k}+1}\le {\text{Δ}}_{\tilde{k}}-{\text{Δ}}_{\tilde{k}+1}+\frac{2\xi }{\delta }\gamma {\left({\text{Δ}}_{\tilde{k}}-{\text{Δ}}_{\tilde{k}+1}\right)}^{\left(1-\theta \right)/\theta }.$

Sequences satisfying such inequalities have been studied in [33] . It is shown that

$\Vert y_k-y^{\text{*}}\Vert \le c_1{\tau }^k.$(3.37)

$\Vert y_k-y^{\text{*}}\Vert \le c_2{k}^{\left(\theta -1\right)/\left(2\theta -1\right)}.$(3.38)

Recalling that

$\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \le L{L}_h\Vert y_{k+1}-y_k\Vert ,$

consequently,

$\Vert {\lambda }_k-{\lambda }^{\text{*}}\Vert \le L{L}_h\Vert y_k-y^{\text{*}}\Vert .$(3.39)

Furthermore, from the relations

${\lambda }_k={\lambda }_{k-1}-\alpha \beta \left(\nabla h\left(A{x}_k-b\right)-\nabla h\left(-y_{k-1}\right)\right)-\beta \left(\nabla h\left(A{x}_k-b\right)-\nabla h\left(-y_k\right)\right),$

and

$\nabla h\left(A{x}^{\text{*}}-b\right)-\nabla h\left(-y^{\text{*}}\right)=0,$

it follows that

$\begin{array}{l}\left(\alpha +1\right)\beta \left(\nabla h\left(A{x}_k-b\right)-\nabla h\left(A{x}^{\text{*}}-b\right)\right)\\ =\beta \left(\nabla h\left(-y_k\right)-\nabla h\left(-y^{\text{*}}\right)\right)+\left({\lambda }_{k-1}-{\lambda }^{\text{*}}\right)+\left({\lambda }^{\text{*}}-{\lambda }_k\right)\\ +\alpha \beta \left(\nabla h\left(-y_{k-1}\right)-\nabla h\left(-y^{\text{*}}\right)\right).\end{array}$

i.e.,

$\begin{array}{l}\left(\nabla h\left(A{x}_k-b\right)-\nabla h\left(A{x}^{\text{*}}-b\right)\right)\\ =\frac{1}{\alpha +1}\left(\nabla h\left(-y_k\right)-\nabla h\left(-y^{\text{*}}\right)\right)+\frac{1}{\left(\alpha +1\right)\beta }\left({\lambda }_{k-1}-{\lambda }^{\text{*}}\right)\\ +\frac{1}{\left(\alpha +1\right)\beta }\left({\lambda }^{\text{*}}-{\lambda }_k\right)+\frac{\alpha }{\alpha +1}\left(\nabla h\left(-y_{k-1}\right)-\nabla h\left(-y^{\text{*}}\right)\right).\end{array}$

Subsequently, combining the 1-strong-convexity of $h(\cdot )$ and above equality, it follows that

$\begin{array}{c}\Vert x_k-x^{\text{*}}\Vert \le \frac{1}{M}\Vert \nabla h\left(A{x}_k-b\right)-\nabla h\left(A{x}^{\text{*}}-b\right)\Vert \\ \le \frac{1}{M}(\frac{1}{\left(\alpha +1\right)\beta }\Vert {\lambda }_{k-1}-{\lambda }^{\text{*}}\Vert +\frac{1}{\left(\alpha +1\right)\beta }\Vert {\lambda }^{\text{*}}-{\lambda }_k\Vert \\ +\frac{1}{\alpha +1}\Vert \nabla h\left(-y_k\right)-\nabla h\left(-y^{\text{*}}\right)\Vert +\frac{\alpha }{\alpha +1}\Vert \nabla h\left(-y_{k-1}\right)-\nabla h\left(-y^{\text{*}}\right)\Vert )\end{array}$

$\begin{array}{c}\le \frac{1}{M}(\frac{1}{\left(\alpha +1\right)\beta }\Vert {\lambda }_{k-1}-{\lambda }^{\text{*}}\Vert +\frac{1}{\left(\alpha +1\right)\beta }\Vert {\lambda }^{\text{*}}-{\lambda }_k\Vert \\ +\frac{L_h}{\left(\alpha +1\right)\beta }\Vert y_k-y^{\text{*}}\Vert +\frac{\alpha L_h}{\left(\alpha +1\right)\beta }\Vert y_{k-1}-y^{\text{*}}\Vert )\\ \le \frac{1}{M}(\frac{L{L}_h}{\left(\alpha +1\right)\beta }\Vert y_{k-1}-y^{\text{*}}\Vert +\frac{L{L}_h}{\left(\alpha +1\right)\beta }\Vert y_k-y^{\text{*}}\Vert \\ +\frac{L_h}{\left(\alpha +1\right)\beta }\Vert y_k-y^{\text{*}}\Vert +\frac{\alpha L_h}{\left(\alpha +1\right)\beta }\Vert y_{k-1}-y^{\text{*}}\Vert )\\ \le \frac{1}{M}\left(\frac{\left(1+L\right)L_h}{\left(\alpha +1\right)\beta }\Vert y_k-y^{\text{*}}\Vert +\frac{\left(\alpha +L\right)L_h}{\left(\alpha +1\right)\beta }\Vert y_{k-1}-y^{\text{*}}\Vert \right).\end{array}$(3.40)

The desired inequalities follow from (3.37)-(3.40) immediately.□

4. Conclusions

In this paper, we proposed a symmetric Bregman ADMM, which can return to the symmetric ADMM, the Bregman ADMM and classical ADMM while circumventing the requirement for global Lipschitz continuity of the gradient when minimizing a linearly constrained nonconvex minimization problem whose objective function is the sum of two separable nonconvex functions. Moreover, we analyze its convergence, under certain assumptions and when the associated function satisfies the Kurdyka-Lojasiewicz inequality, we prove that the iterative sequence generated by the symmetric Bregman ADMM converges to a critical point of the problem. Finally, we establish the convergence rate of the algorithm.

References

1. Goldstein, T., O’Donoghue, B., Setzer, S. and Baraniuk, R. (2014) Fast Alternating Direction Optimization Methods. SIAM Journal on Imaging Sciences, 7, 1588-1623. >https://doi.org/10.1137/120896219

2. Boyd, S. (2010) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends in Machine Learning, 3, 1-122. >https://doi.org/10.1561/2200000016

3. Yin, W., Osher, S., Goldfarb, D. and Darbon, J. (2008) Bregman Iterative Algorithms for-Minimization with Applications to Compressed Sensing. SIAM Journal on Imaging Sciences, 1, 143-168. >https://doi.org/10.1137/070703983

4. Figueiredo, M.A.T. and Bioucas-Dias, J.M. (2010) Restoration of Poissonian Images Using Alternating Direction Optimization. IEEE Transactions on Image Processing, 19, 3133-3145. >https://doi.org/10.1109/tip.2010.2053941

5. Goldstein, T., Bresson, X. and Osher, S. (2009) Geometric Applications of the Split Bregman Method: Segmentation and Surface Reconstruction. Journal of Scientific Computing, 45, 272-293. >https://doi.org/10.1007/s10915-009-9331-z

6. Ochs, P., Chen, Y., Brox, T. and Pock, T. (2014) iPiano: Inertial Proximal Algorithm for Nonconvex Optimization. SIAM Journal on Imaging Sciences, 7, 1388-1419. >https://doi.org/10.1137/130942954

7. Candes, E.J. and Tao, T. (2006) Near-Optimal Signal Recovery from Random Projections: Universal Encoding Strategies? IEEE Transactions on Information Theory, 52, 5406-5425. >https://doi.org/10.1109/tit.2006.885507

8. Beck, A. and Teboulle, M. (2009) Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems. IEEE Transactions on Image Processing, 18, 2419-2434. >https://doi.org/10.1109/tip.2009.2028250

9. Dai, W. and Milenkovic, O. (2009) Subspace Pursuit for Compressive Sensing Signal Reconstruction. IEEE Transactions on Information Theory, 55, 2230-2249. >https://doi.org/10.1109/tit.2009.2016006

10. Aharon, M., Elad, M. and Bruckstein, A. (2006) K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Transactions on Signal Processing, 54, 4311-4322. >https://doi.org/10.1109/tsp.2006.881199

11. Gabay, D. and Mercier, B. (1976) A Dual Algorithm for the Solution of Nonlinear Variational Problems via Finite Element Approximation. Computers & Mathematics with Applications, 2, 17-40. >https://doi.org/10.1016/0898-1221(76)90003-1

12. Glowinski, R. and Marroco, A. (1975) Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Revue française d’automatique, informatique, recherche opérationnelle. Analyse numérique, 9, 41-76. >https://doi.org/10.1051/m2an/197509r200411

13. Boley, D. (2013) Local Linear Convergence of the Alternating Direction Method of Multipliers on Quadratic or Linear Programs. SIAM Journal on Optimization, 23, 2183-2207. >https://doi.org/10.1137/120878951

14. Han, D. and Yuan, X. (2013) Local Linear Convergence of the Alternating Direction Method of Multipliers for Quadratic Programs. SIAM Journal on Numerical Analysis, 51, 3446-3457. >https://doi.org/10.1137/120886753

15. He, B. and Yuan, X. (2012) On the Convergence Rate of the Douglas-Rachford Alternating Direction Method. SIAM Journal on Numerical Analysis, 50, 700-709. >https://doi.org/10.1137/110836936

16. Yang, W.H. and Han, D. (2016) Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems. SIAM Journal on Numerical Analysis, 54, 625-640. >https://doi.org/10.1137/140974237

17. Li, G. and Pong, T.K. (2015) Global Convergence of Splitting Methods for Nonconvex Composite Optimization. SIAM Journal on Optimization, 25, 2434-2460. >https://doi.org/10.1137/140998135

18. Hong, M., Luo, Z. and Razaviyayn, M. (2016) Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems. SIAM Journal on Optimization, 26, 337-364. >https://doi.org/10.1137/140990309

19. Guo, K., Han, D.R. and Wu, T.T. (2016) Convergence of Alternating Direction Method for Minimizing Sum of Two Nonconvex Functions with Linear Constraints. International Journal of Computer Mathematics, 94, 1653-1669. >https://doi.org/10.1080/00207160.2016.1227432

20. Wu, Z., Li, M., Wang, D.Z.W. and Han, D. (2017) A Symmetric Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problems. Asia-Pacific Journal of Operational Research, 34, Article ID: 1750030. >https://doi.org/10.1142/s0217595917500300

21. Bauschke, H.H., Bolte, J. and Teboulle, M. (2017) A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications. Mathematics of Operations Research, 42, 330-348. >https://doi.org/10.1287/moor.2016.0817

22. Dragomir, R., d’Aspremont, A. and Bolte, J. (2021) Quartic First-Order Methods for Low-Rank Minimization. Journal of Optimization Theory and Applications, 189, 341-363. >https://doi.org/10.1007/s10957-021-01820-3

23. Nesterov, Y. (2019) Implementable Tensor Methods in Unconstrained Convex Optimization. Mathematical Programming, 186, 157-183. >https://doi.org/10.1007/s10107-019-01449-1

24. Tan, L. and Guo, K. (2025) Bregman ADMM: A New Algorithm for Nonconvex Optimization with Linear Constraints. Journal of Nonlinear and Variational Analysis, 9, 176-196. >https://doi.org/10.23952/jnva.9.2025.2.02

25. Wang, H. and Banerjee, A. (2014) Bregman Alternating Direction Method of Multipliers. arXiv: 1306.3203.

26. Beck, A. (2017) First-Order Methods in Optimization. Society for Industrial and Applied Mathematics. >https://doi.org/10.1137/1.9781611974997

27. Bolte, J., Sabach, S., Teboulle, M. and Vaisbourd, Y. (2018) First Order Methods Beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems. SIAM Journal on Optimization, 28, 2131-2151. >https://doi.org/10.1137/17m1138558

28. Attouch, H., Bolte, J., Redont, P. and Soubeyran, A. (2010) Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality. Mathematics of Operations Research, 35, 438-457. >https://doi.org/10.1287/moor.1100.0449

29. Attouch, H., Bolte, J. and Svaiter, B.F. (2011) Convergence of Descent Methods for Semi-Algebraic and Tame Problems: Proximal Algorithms, Forward-Backward Splitting, and Regularized Gauss–seidel Methods. Mathematical Programming, 137, 91-129. >https://doi.org/10.1007/s10107-011-0484-9

30. Bolte, J., Sabach, S. and Teboulle, M. (2013) Proximal Alternating Linearized Minimization for Nonconvex and Nonsmooth Problems. Mathematical Programming, 146, 459-494. >https://doi.org/10.1007/s10107-013-0701-9

31. Bauschke, H.H., Borwein, J.M. and Combettes, P.L. (2003) Bregman Monotone Optimization Algorithms. SIAM Journal on Control and Optimization, 42, 596-636. >https://doi.org/10.1137/s0363012902407120

32. Bregman, L.M. (1967) The Relaxation Method of Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming. USSR Computational Mathematics and Mathematical Physics, 7, 200-217. >https://doi.org/10.1016/0041-5553(67)90040-7

33. Attouch, H. and Bolte, J. (2007) On the Convergence of the Proximal Algorithm for Nonsmooth Functions Involving Analytic Features. Mathematical Programming, 116, 5-16. >https://doi.org/10.1007/s10107-007-0133-5