Convergence of Generalized Bregman Alternating Direction Method of Multipliers for Nonconvex Objective with Linear Constraints

In this paper, we investigate the convergence of the generalized Bregman alternating direction method of multipliers (ADMM) for solving nonconvex separable problems with linear constraints. This algorithm relaxes the requirement of global Lipschitz continuity of differentiable functions that is often seen in many researches, and it incorporates the acceleration technique of the proximal point algorithm (PPA). As a result, the scope of application of the algorithm is broadened and its performance is enhanced. Under the assumption that the augmented Lagrangian function satisfies the Kurdyka-Lojasiewicz inequality, we demonstrate that the iterative sequence generated by the algorithm converges to a critical point of its augmented Lagrangian function when the penalty parameter in the augmented Lagrangian function is sufficiently large. Finally, we analyze the convergence rate of the algorithm.

• Generalized Bregman Alternating Direction Method of Multipliers
• Nonconvex Optimization
• Lipschitz-Like Convexity Condition
• Kurdyka-Lojasiewicz Inequality

1. Introduction

In this paper, what we consider is the two-block separable optimization problem model with linear constraints:

$\begin{array}{l}\underset{x,y}{\text{min}}f\left(x\right)+g\left(y\right),\\ \text{s}\text{.t}\text{.}Ax+y=b,\end{array}$ (1.1)

where $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is a proper lower semicontinuous function, $g:{ℝ}^m\to ℝ$ is a continuously differentiable function, $A\in {ℝ}^{m\times n}$ is a matrix, and $b\in {ℝ}^m$ is a vector. Many valuable optimization problems can be formulated in the form of (1.2), making it applicable across a wide range of fields, such as image and signal processing [1] - [4] , statistical learning [5] , and compressed sensing [6] [7] .

Among the many methods to solve this kind of problem (1.1), the Alternating Direction Method of Multipliers (ADMM) is one of the most classic methods. The iterative scheme of the ADMM is as follows:

$\{\begin{array}{l}{x}^{k+1}=\underset{x}{\arg \min }\left\{{ℒ}_{\beta }\left(x,y^k,{\lambda }^k\right)\right\},\\ y^{k+1}=\underset{y}{\arg \min }\left\{{ℒ}_{\beta }\left(x^{k+1},y,{\lambda }^k\right)\right\},\\ {\lambda }^{k+1}={\lambda }^k+\beta \left(A{x}^{k+1}+y^{k+1}-b\right).\end{array}$ (1.2)

Here, ${ℒ}_{\beta }(\cdot )$ denotes the augmented Lagrangian function for (1.1):

${ℒ}_{\beta }\left(x,y,\lambda \right)=f\left(x\right)+g\left(y\right)+\langle \lambda ,Ax+y-b\rangle +\frac{\beta }{2}{\Vert Ax+y-b\Vert }^2,$

where $\lambda$ is the Lagrangian multiplier associated with the linear constraint, and $\beta >0$ is the penalty parameter. ADMM has been known since the mid-1970s when it was introduced by Gabay, Mercier, Glowinski, and Marrocco [8] [9] . When both $f$ and $g$ are convex functions, ADMM has produced a number of well-understood results for both convergence and rate of convergence in problem (1.1) [10] - [15] . However, when the objective function contains a nonconvex part, many subsequent studies have focused on variants of ADMM, often adding conditions to prove the corresponding convergence. For instance, Li and Pong [16] proposed a proximal ADMM, and in their convergence analysis, they required the constraint to be $Ax=y$. Subsequently, Hong, Luo, and Razaviyayn [17] proved the convergence of ADMM for solving consensus and sharing problems by assuming that the penalty parameter in the augmented Lagrangian is chosen to be sufficiently large (in 2016). In 2017, Guo et al. [18] further improved on these results, demonstrating that under more concise conditions than those in [16] [17] , provided that the augmented Lagrangian function satisfies the Kurdyka-Lojasiewicz inequality.

Reference [19] indicates that ADMM (1.2) is actually the dual of the well-known Douglas-Rachford splitting method (DRSM) [20] applied to problem (1.1). In the literature [21] , DRSM is further interpreted as a special case of the proximity point algorithm (PPA). Additionally, literature [21] uses the acceleration form of PPA to accelerate the original ADMM (1.2). With this acceleration technique, Guo et al. proposed a Generalized Alternating Direction Method of Multipliers (GADMM) in 2018 to accelerate the original ADMM for both separable and inseparable problems [22] . The iteration format for one of them is as follows:

$\{\begin{array}{l}{x}^{k+1}\in \underset{x}{\arg \min }\left\{f\left(x\right)+H\left(x,y^k\right)-\langle {\lambda }^k,Ax\rangle +\frac{\beta }{2}{\Vert Ax+y^k-b\Vert }^2\right\}\\ y^{k+1}\in \underset{y}{\arg \min }\left\{g\left(y\right)+H\left(x^{k+1},y\right)-\langle {\lambda }^k,y\rangle +\frac{\beta }{2}{\Vert \alpha A{x}^{k+1}+\left(1-\alpha \right)\left(b-y^k\right)+y-b\Vert }^2\right\}\\ {\lambda }^{k+1}={\lambda }^k-\beta \left(\alpha A{x}^{k+1}+\left(1-\alpha \right)\left(b-y^k\right)+y^{k+1}-b\right)\end{array}$ (1.3)

Obviously, the GADMM (1.3) reduces to the classic ADMM (1.2) when $\alpha =1$, and it reduces to the classic GADMM [21] when $H\equiv 0$. We can prove the acceleration performance of the GADMM [21] through references [23] - [25] . Nevertheless, investigations into the application of ADMM within the realm of nonconvex optimization frequently hinge on the premise that the gradient of the differentiable function adheres to global Lipschitz continuity. However, this condition is not universally satisfied in pivotal problem formulations across domains such as Poisson inverse problems [26] , quadratic inverse problems [27] , and rank minimization [28] [29] . Consequently, this stringent assumption unduly constrains the versatility of ADMM in these critical areas.

In 2016, Bauschke, Bolte, and Teboulle [26] introduced the Lipschitz-like convexity condition as a relaxation of the global Lipschitz continuous gradient assumption in optimization problems. In 2018, Bolte, Sabach, and Teboulle [27] further introduced the $L$-smooth adaptivity condition as a supplement to the Lipschitz-like convexity condition. These conditions allowed ADMM to be extended to problems where the gradient of the differentiable function does not satisfy global Lipschitz continuity. Recently, Guo and Tan [30] proposed a “real” Bregman ADMM based on these weakened conditions, which can reduce to the classical ADMM and override its results [18] . The iterative format for Guo and Tan’s Bregman ADMM is as follows:

$\{\begin{array}{l}{x}^{k+1}=\underset{x}{\arg \min }\left\{{ℒ}_{\beta }^h\left(x,y^k,{\lambda }^k\right)\right\}\\ y^{k+1}=\underset{y}{\arg \min }\left\{{ℒ}_{\beta }^h\left(x^{k+1},y,{\lambda }^k\right)\right\}\\ {\lambda }^{k+1}={\lambda }^k+\beta \left(\nabla h\left(A{x}^{k+1}-b\right)-\nabla h\left(-y^{k+1}\right)\right)\end{array}$ (1.4)

where ${ℒ}_{\beta }^h(\cdot )$ denotes the Bregman augmented Lagrangian function for (1.1):

${ℒ}_{\beta }^h\left(x,y,\lambda \right)=f\left(x\right)+g\left(y\right)+\langle \lambda ,Ax+y-b\rangle +\beta D_h\left(-y,Ax-b\right).$ (1.5)

When $h(\cdot )=\frac{1}{2}{\Vert \cdot \Vert }^2$, the Bregman ADMM (1.4) reduces to the classical ADMM (1.2).

Combining the aforementioned content, we aim to integrate the acceleration technique from the Proximal Point Algorithm (PPA) while relaxing the requirement of gradient Lipschitz continuity for differentiable functions in ADMM. To this end, we propose a generalized Bregman ADMM, whose iterative format is as follows:

$\{\begin{array}{l}{x}^{k+1}=\underset{x}{\arg \min }\left\{f\left(x\right)+g\left(y^k\right)-\langle {\lambda }^k,Ax+y^k-b\rangle +\frac{\beta }{2}{\Vert Ax+y^k-b\Vert }^2\right\},\\ y^{k+1}=\underset{y}{\arg \min }\left\{f\left(x^{k+1}\right)+g\left(y\right)-\langle {\lambda }^k,A{x}^{k+1}+y-b\rangle +\beta D_h\left(-y-\left(\alpha -1\right)y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\right\},\\ {\lambda }^{k+1}={\lambda }^k-\beta \left(\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\right).\end{array}$ (1.6)

Here, the parameter $\alpha$ is the relaxation factor. The generalized Bregman ADMM (1.6) elegantly transitions to the generalized ADMM (1.3) by setting $h(\cdot )=\frac{1}{2}{\Vert \cdot \Vert }^2$, and to the Bregman ADMM (1.4) by setting $\alpha =1$.

Remark 1.1 It is worth noting that, since we have only weakened the conditions on the function $g\left(y\right)$, the iterative format for the variable $x$ remains unchanged (consistent with the classical method). Only the Bregman distance $D_h$ has been introduced in the iterative format for the variable $y$.

We know that a very important technique for proving the convergence of nonconvex optimization problems depends on assuming that the objective function satisfies the Kurdyka-Lojasiewicz (KL) inequality. This assumption is also used in many previous articles [16] [18] [30] . Therefore, we also assume that the function satisfies the KL inequality. It is further proved that when the augmented Lagrangian function is a KL function, the sequence generated by the generalized Bregman ADMM converges to a KKT point of the problem (1.1). Ultimately, we analyze the convergence rate of the proposed algorithm under the specified parameter configurations. The structure of the remainder of this paper is outlined as follows. In Section 2, we establish the necessary theoretical foundations for our subsequent analysis. In Section 3, we conduct a detailed convergence analysis of the generalized Bregman Alternating Direction Method of Multipliers (ADMM) and determine its convergence rate. Lastly, in Section 4, we encapsulate our key findings and present our conclusions.

2. Preliminaries

In this section, we review some definitions and fundamental results that will be utilized in our subsequent analysis.

Definition 2.1 [31] For an extended real-valued function $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$, the effective domain, or simply the domain, is the set

$dom\left(f\right)=\left\{x\in {ℝ}^n:f\left(x\right)<\infty \right\}.$

Definition 2.2 [31] A function $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is called proper if there exists at least one $x\in {ℝ}^n$ such that $f\left(x\right)<\infty$.

Definition 2.3 [31] A function $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is called lower semicontinuous at $x\in {ℝ}^n$ if

$f\left(x\right)\le \underset{k\to \infty }{\lim \inf }f\left(x^k\right)$

for any sequence $\left\{x^k\right\}\subseteq {ℝ}^n$ such that $x_k\to x$ as $k\to \infty$. Moreover, $f$ is called lower semicontinuous if it is lower semicontinuous at each point in ${ℝ}^n$.

Definition 2.4 ( [27] , kernel generating distance) Let $C$ be a nonempty, convex, and open subset of ${ℝ}^m$. A function $h:{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$ associated with $C$ is called a kernel generating distance if it satisfies the following conditions:

i) $h$ is proper, lower semicontinuous, and convex, with $dom\left(h\right)\subset \bar{C}$ and $dom\left(\partial h\right)=C$.

ii) $h$ is $C^1$ on $int\left(dom\left(h\right)\right)\equiv C$.

We denote the class of kernel generating distances by $\mathcal{G}\left(C\right)$.

Definition 2.5 ( [32] ) Let $h\in \mathcal{G}\left(C\right)$. The Bregman distance $D_h:dom\left(h\right)\times int\left(dom\left(h\right)\right)\to {ℝ}^{+}$ is defined by

$D_h\left(x,y\right):=h\left(x\right)-h\left(y\right)-\langle \nabla h\left(y\right),x-y\rangle .$ (2.1)

Since $h$ is convex, $D_h\left(x,y\right)\ge 0$, and $D_h\left(x,y\right)=0$ if and only if $x=y$.

Lemma 2.1 ( [33] ) Let $h\in \mathcal{G}\left(C\right)$. For any $x,y\in int\left(dom\left(h\right)\right)$ and $z\in dom\left(h\right)$, the following properties hold:

i) $D_h\left(x,y\right)+D_h\left(y,x\right)=\langle \nabla h\left(x\right)-\nabla h\left(y\right),x-y\rangle$.

ii) The three-point identity holds:

$D_h\left(z,x\right)-D_h\left(z,y\right)-D_h\left(y,x\right)=\langle \nabla h\left(x\right)-\nabla h\left(y\right),y-z\rangle .$

Definition 2.6 ( [27] , L-smooth adaptable) Let $h\in \mathcal{G}\left(C\right)$ and $g:{ℝ}^m\to ℝ$ be continuously differentiable on $C=int\left(dom\left(h\right)\right)$. A pair $\left(g,h\right)$ is called L-smooth adaptable on $C$ if there exists $L>0$ such that $Lh-g$ and $Lh+g$ are convex on $C$.

Remark 2.1 Definition 2.6 naturally complements and extends the definition of “A Lipschitz-like/Convexity Condition” in [26] , which allows us to obtain the following two-sided descent lemma.

Lemma 2.2 ( [27] , extended descent lemma) The pair of functions $\left(g,h\right)$ is L-smooth adaptable on $C$ if and only if

$\left|g\left(x\right)-g\left(y\right)-\langle \nabla g\left(y\right),x-y\rangle \right|\le L{D}_h\left(x,y\right)\forall x,y\in int\left(dom\left(h\right)\right).$ (2.2)

Remark 2.2 In particular, when the set $C={ℝ}^m$ and $h(\cdot )=\frac{1}{2}{\Vert \cdot \Vert }^2$, (2.2) reduces to the classical descent lemma for the function $g$, i.e.,

$\left|g\left(x\right)-g\left(y\right)-\langle \nabla g\left(y\right),x-y\rangle \right|\le \frac{L}{2}\cdot {\Vert x-y\Vert }^2\forall x,y\in {ℝ}^m.$

Definition 2.7 ( [26] ) Let $g:{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$ be a proper and lower semicontinuous function. The gradient of $g$ is D-Lipschitz if there exists $L>0$ satisfying

$\Vert \nabla g\left(x\right)-\nabla g\left(y\right)\Vert \le L\frac{D_h\left(x,y\right)+D_h\left(y,x\right)}{\Vert x-y\Vert },x\ne y\in int\left(dom\left(h\right)\right).$

Remark 2.3 According to the Cauchy-Schwarz inequality, we have

$\left|\langle \nabla g\left(x\right)-\nabla g\left(y\right),x-y\rangle \right|\le \Vert \nabla g\left(x\right)-\nabla g\left(y\right)\Vert \Vert x-y\Vert ,$

which, combined with Definition 2.7, yields

$\left|\langle \nabla g\left(x\right)-\nabla g\left(y\right),x-y\rangle \right|\le L\left(D_h\left(x,y\right)+D_h\left(y,x\right)\right).$

Using the conclusion in Lemma 2.1, the above inequality is equivalent to

$\langle \nabla g\left(x\right)-\nabla g\left(y\right),x-y\rangle \le L\left(D_h\left(x,y\right)+D_h\left(y,x\right)\right)=L\langle \nabla h\left(x\right)-\nabla h\left(y\right),x-y\rangle ,$

$\langle \nabla g\left(y\right)-\nabla g\left(x\right),x-y\rangle \le L\left(D_h\left(x,y\right)+D_h\left(y,x\right)\right)=L\langle \nabla h\left(x\right)-\nabla h\left(y\right),x-y\rangle .$

Thus,

$\langle \left(L\nabla h\left(x\right)-\nabla g\left(x\right)\right)-\left(L\nabla h\left(y\right)-\nabla g\left(y\right)\right),x-y\rangle \ge 0,$

$\langle \left(L\nabla h\left(x\right)+\nabla g\left(x\right)\right)-\left(L\nabla h\left(y\right)+\nabla g\left(y\right)\right),x-y\rangle \ge 0.$ (2.3)

According to inequality (2.3), the functions $Lh+g$ and $Lh-g$ are convex functions since their gradients are monotone on the set $C$. Therefore, the D-Lipschitz continuity property of the gradient of the function $g$ is a sufficient condition for the function pair $\left(g,h\right)$ to be L-smooth adaptable. In this paper, given the complexity of the iterative scheme of ADMM, we need to assume that the gradient of $g$ is D-Lipschitz continuous.

Remark 2.4 Indeed, the $D$-Lipschitz continuity property is a sort of Lipschitz-like gradient property of the function $g(\cdot )$ with respect to the Bregman distance, which reduces to gradient Lipschitz continuity of function $g(\cdot )$ when $h(\cdot )=\frac{1}{2}{\Vert \cdot \Vert }^2$.

Definition 2.8 [18] Let $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper lower semicontinuous function.

i) The Fréchet subdifferential, or regular subdifferential, of $f(\cdot )$ at $x\in dom\left(f\right)$, written $\hat{\partial }f\left(x\right)$, is the set of vectors $x^{\text{*}}\in {ℝ}^n$ that satisfy

$\underset{y\ne x}{\text{lim}}\underset{y\to x}{\text{inf}}\frac{f\left(y\right)-f\left(x\right)-\langle x^{\text{*}},y-x\rangle }{\Vert y-x\Vert }\ge 0.$

When $x\notin dom\left(f\right)$, we set $\hat{\partial }f\left(x\right)=\varnothing$.

ii) The limiting-subdifferential, or simply the subdifferential, of $f(\cdot )$ at $x\in dom\left(f\right)$, written $\partial f\left(x\right)$, is defined as follows:

$\partial f\left(x\right)=\left\{x^{\text{*}}\in {ℝ}^n,\exists x_n\to x,f\left(x_n\right)\to f\left(x\right),x_n^{\text{*}}\in \hat{\partial }f\left(x_n\right),\text{with}{x}_n^{\text{*}}\to x^{\text{*}}\right\}.$

Remark 2.5 From the above definition, we note that

i) It implies that $\hat{\partial }f\left(x\right)\subseteq \partial f\left(x\right)$ for each $x\in {ℝ}^n$, where the first set is closed convex while the second one is only closed.

ii) Let $\left(x_k,x_k^{\text{*}}\right)\in Graph\partial f$ be a sequence that converges to $\left(x,x^{\text{*}}\right)$. By the definition of $\partial f$, if $f\left(x_k\right)$ converges to $f\left(x\right)$ as $k\to +\infty$, then $\left(x,x^{\text{*}}\right)\in Graph\partial f$, where $Graph\partial f=\left\{\left(x,y\right)|y\in \partial f\left(x\right)\right\}$.

iii) A necessary condition for $x\in {ℝ}^n$ to be a minimizer of $f(\cdot )$ is

$0\in \partial f\left(x\right).$ (2.4)

iv) If $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is a proper lower semicontinuous and $g:{ℝ}^n\to ℝ$ is continuous differentiable, then $\partial \left(f+g\right)\left(x\right)=\partial f\left(x\right)+\nabla g\left(x\right)$ for any $x\in dom\left(f\right)$.

A point satisfying (2.4) is called a critical point or a stationary point. The critical points set of $f$ is denoted by $\text{crit}f$.

Now, we recall an important property of subdifferential calculus.

Lemma 2.3 [34] Suppose that $F\left(x,y\right)=f\left(x\right)+g\left(x\right)$, where $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ and $g:{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$ are proper lower semicontinuous functions. Then for all $\left(x,y\right)\in dom\left(F\right)=dom\left(f\right)\times dom\left(g\right)$, we have

$\partial F\left(x,y\right)={\partial }_{x}F\left(x,y\right)\times {\partial }_{y}F\left(x,y\right).$

Definition 2.9 ( [34] , Kurdyka-Lojasiewicz inequality) Let $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper lower semicontinuous function. For $-\infty <{\eta }_1<{\eta }_2\le +\infty$, set

$\left[{\eta }_1<f<{\eta }_2\right]=\left\{x\in {ℝ}^n:{\eta }_1<f\left(x\right)<{\eta }_2\right\}.$

We say that function $f(\cdot )$ has the KL property at $x^{\text{*}}\in dom\left(\partial f\right)$ if there exist $\eta \in \left(0,+\infty \right]$, a neighbourhood $U$ of $x^{\text{*}}$, and a continuous concave function $\phi :\left[0,\eta \right)\to {ℝ}_{+}$, such that

i) $\phi \left(0\right)=0$;

ii) $\phi$ is $C^1$ on $\left(0,\eta \right)$ and continuous at 0;

iii) ${\phi }^{\prime }\left(x\right)>0,\forall x\in \left(0,\eta \right)$;

iv) for all $x$ in $U\cap \left[f\left(x^{\text{*}}\right)<f<f\left(x^{\text{*}}\right)+\eta \right]$, the Kurdyka-Lojasiewicz inequality holds

${\phi }^{\prime }\left(f\left(x\right)-f\left(x^{\text{*}}\right)\right)d\left(0,\partial f\left(x\right)\right)\ge 1,$

where $d\left(x,\partial f\left(x\right)\right)=\underset{y\in \partial f\left(x\right)}{\text{inf}}\Vert y-x\Vert$, is the distance from $x$ to $\partial f\left(x\right)$.

Remark 2.6 Denote ${\Phi }_{\eta }$ be the set of all continuous functions $\phi (\cdot )$ which satisfy (i) - (iii).

Definition 2.10 ( [35] , Kurdyka-Lojasiewicz function) If $f(\cdot )$ satisfies the KL property at each point of $dom\left(\partial f\right)$, then $f(\cdot )$ is called a KL function.

Lemma 2.4 ( [36] , Uniformized KL property) Let $\Omega$ be a compact set and $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper and lower semicontinuous function. Assume that $f(\cdot )$ is constant on $\Omega$ and satisfies the KL property at each point of $\Omega$. Then, there exist $ϵ>0,\eta >0$, and $\phi \in {\Phi }_{\eta }$ such that for all $\bar{x}\in \Omega$ and for all $x$ in the following intersection:

$\left\{x\in {ℝ}^n:d\left(x,\text{Ω}\right)<ϵ\right\}\cap \left[f\left(\bar{x}\right)<f<f\left(\bar{x}\right)+\eta \right],$

one has

${\phi }^{\prime }\left(f\left(x\right)-f\left(\bar{x}\right)\right)d\left(0,\partial f\left(x\right)\right)\ge 1.$

Definition 2.11 We say that $\left(x^{*},y^{*},{\lambda }^{*}\right)$ is a critical point of the Augmented Lagrangian Function with Bregman distance ${ℒ}_{\beta }^h(\cdot )$ (3.3) if it satisfies

$\{\begin{array}{l}-A^{\text{T}}{\lambda }^{\text{*}}\in \partial f\left(x^{\text{*}}\right),\\ -{\lambda }^{\text{*}}=\nabla g\left(y^{\text{*}}\right),\\ A{x}^{\text{*}}-y^{\text{*}}=b.\end{array}$

3. Convergence Analysis

To ensure that the Generalized Bregman ADMM (1.6) is well-defined and generates an infinite iterative sequence $\left\{\left(x^k,y^k,{\lambda }^k\right)\right\}$, we assume that the two minimization subproblems in (1.6) have solutions throughout the analysis. The optimality conditions for (1.6) are:

$\{\begin{array}{l}0\in \partial f\left(x^{k+1}\right)-A^{\text{T}}{\lambda }^k+\beta A^{\top }\left(A{x}^{k+1}+y^k-b\right)\\ 0=\nabla g\left(y^{k+1}\right)-{\lambda }^k+\beta \left(\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\right)\\ {\lambda }^{k+1}={\lambda }^k-\beta \left(\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\right)\end{array}$ (3.1)

In terms of rearrangement of (3.1), it is equivalent to the following relation:

$\{\begin{array}{ll}{A}^{\text{T}}{\lambda }^k-\beta A^{\top }\left(A{x}^{k+1}+y^k-b\right)\in \partial f\left(x^{k+1}\right)\hfill & (3.2\text{a})\hfill \\ {\lambda }^k-\beta \left(\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\right)=\nabla g\left(y^{k+1}\right)\hfill & (3.2\text{b})\hfill \\ {\lambda }^{k+1}={\lambda }^k-\beta \left(\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\right).\hfill & (3.2\text{c})\hfill \end{array}$

To analyze the Generalized Bregman ADMM (1.6), we make the following basic assumptions. Assumption A. Assuming that $f:{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is a proper lower semicontinuous function, $g:{ℝ}^m\to ℝ$ is a continuously differentiable function with $\nabla g$ being $D$-Lipschitz continuous and $h\in \mathcal{G}\left(C\right)$ is a twice differentiable function function on $C=int\left(dom\left(h\right)\right)$, 1-strong-convex, and $\nabla h$ is Lipschitz continuous with $L_h$ on any bounded subset of ${ℝ}^m$. Assume the following conditions hold:

i) if $\alpha \in \left(\frac{L_h+L_h^2}{1+L_h+L_h^2},1\right]$, then $\beta \ge \frac{7L{L}_h^3+\sqrt{49L^2{L}_h^6+16L^2{L}_h^3}}{\alpha \left(4+4L_h+4L_h^2\right)-\left(4L_h+4L_h^2\right)}$ which implies

$\delta =\left[\frac{2\beta }{L_h}-\frac{2\beta \left(1-\alpha \right)\left(1+L_h\right)}{\alpha }-\frac{7L{L}_h^2}{\alpha }-\frac{2L^2{L}_h^2}{\alpha \beta }\right]\ge 0$

ii) if $\alpha \in \left[1,\frac{-L_h-L_h^2}{1-L_h-L_h^2}\right)$, then $\beta \ge \frac{\left(\alpha +6\right)L{L}_h^3+\sqrt{{\left(\left(\alpha +6\right)L{L}_h^3\right)}^2+16\alpha L^2{L}_h^3}}{\alpha \left(4-4L_h-4L_h^2\right)+\left(4L_h+4L_h^2\right)}$ which implies

$\delta =\left[\frac{2\beta }{L_h}-\frac{2\beta \left(\alpha -1\right)\left(1+L_h\right)}{\alpha }-\frac{\left(\alpha +6\right)L{L}_h^2}{\alpha }-\frac{2L^2{L}_h^2}{\alpha \beta }\right]\ge 0$

iii) $A^{\text{T}}A\underset{\_}{\succ }MI$ for some $M>0$.

The Bregman augmented Lagrangian function of problem (1.1) is defined by

${ℒ}_{\beta }^h\left(x,y,\lambda \right)=f\left(x\right)+g\left(y\right)-\langle \lambda ,Ax+y-b\rangle +\frac{\beta }{2}{\Vert Ax+y-b\Vert }^2+\beta D_h\left(-y,Ax-b\right).$ (3.3)

Here, $\lambda$ is the Lagrangian multiplier associated with the linear constraints, and $\beta >0$ is the penalty parameter. Moreover, we set

$\begin{array}{c}{\widehat{ℒ}}_{\beta }^h\left(x,y,\lambda ;\alpha ,v\right):=f\left(x\right)+g\left(y\right)-\langle \lambda ,Ax+y-b\rangle +\frac{\beta }{2}{\Vert Ax+y-b\Vert }^2\\ +\beta D_h\left(-y-\left(\alpha -1\right)v,\alpha \left(Ax-b\right)\right).\end{array}$ (3.4)

Now, we begin our analysis with the following technical lemma.

Lemma 3.1 Let $\left\{w^k=\left(x^k,y^k,{\lambda }^k\right)\right\}$ be the sequence generated by the Generalized Bregman ADMM (1.6), which is assumed to be bounded. Then we have

${ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{k+1}\right)\ge \delta D_h\left(y^k,y^{k+1}\right).$ (3.5)

Proof. From the definition of ${\widehat{ℒ}}_{\beta }^h$ in (3.4), it follows that

$\begin{array}{l}{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;\alpha ,y^k\right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^k;\alpha ,y^k\right)\\ =f\left(x^{k+1}\right)+g\left(y^k\right)-\langle {\lambda }^k,A{x}^{k+1}+y^k-b\rangle +\beta D_h\left(-\alpha y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\\ +\frac{\beta }{2}{\Vert A{x}^{k+1}+y^k-b\Vert }^2-[f\left(x^{k+1}\right)+g\left(y^{k+1}\right)-\langle {\lambda }^k,A{x}^{k+1}+y^{k+1}-b\rangle \begin{array}{c}\\ \end{array}\\ +\beta D_h\left(-y^{k+1}-\left(\alpha -1\right)y^k,\alpha \left(A{x}^{k+1}-b\right)\right)+\frac{\beta }{2}{\Vert A{x}^{k+1}+y^{k+1}-b\Vert }^2]\\ =g\left(y^k\right)-g\left(y^{k+1}\right)-\langle {\lambda }^k,y^k-y^{k+1}\rangle +\beta D_h\left(-\alpha y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\\ +\frac{\beta }{2}{\Vert A{x}^{k+1}+y^k-b\Vert }^2-\beta D_h\left(-y^{k+1}-\left(\alpha -1\right)y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\\ -\frac{\beta }{2}{\Vert A{x}^{k+1}+y^{k+1}-b\Vert }^2\end{array}$ (3.6)

Given that the gradient of the function $g$ is $D$-Lipschitz continuous within $int\left(dom\left(h\right)\right)$, it can be inferred that the function pair $\left(g,h\right)$ possesses $L$-smooth adaptability. Consequently, by referring to Lemma2.2, we are able to deduce a certain result

$g\left(y^k\right)-g\left(y^{k+1}\right)\ge \langle \nabla g\left(y^{k+1}\right),y^k-y^{k+1}\rangle -L{D}_h\left(y^k,y^{k+1}\right).$ (3.7)

Utilizing the optimal condition (3.2) ${\lambda }^{k+1}=\nabla g\left(y^{k+1}\right)$, and by inequality (3.7) into identity (3.6), we achieve a certain outcome

$\begin{array}{l}{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;\alpha ,y^k\right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^k;\alpha ,y^k\right)\\ =\langle {\lambda }^{k+1}-{\lambda }^k,y^k-y^{k+1}\rangle -L{D}_h\left(y^k,y^{k+1}\right)+\frac{\beta }{2}{\Vert A{x}^{k+1}+y^k-b\Vert }^2\\ -\frac{\beta }{2}{\Vert A{x}^{k+1}+y^{k+1}-b\Vert }^2+\beta D_h\left(-\alpha y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\\ -\beta D_h\left(-y^{k+1}-\left(\alpha -1\right)y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\\ -\beta D_h\left(-\alpha y^k,-y^{k+1}-\left(\alpha -1\right)y^k\right)+\beta D_h\left(-\alpha y^k,-y^{k+1}-\left(\alpha -1\right)y^k\right)\\ \ge -L{D}_h\left(y^k,y^{k+1}\right)+\beta D_h\left(-\alpha y^k,-y^{k+1}-\left(\alpha -1\right)y^k\right)+\frac{\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2\\ +\beta \langle A{x}^{k+1}+y^{k+1}-b,y^k-y^{k+1}\rangle \end{array}$ (3.8)

In which the final inequality is established by applying the three-point identity and (3.9).

$\begin{array}{l}\frac{\beta }{2}{\Vert A{x}^{k+1}+y^k-b\Vert }^2-\frac{\beta }{2}{\Vert A{x}^{k+1}+y^{k+1}-b\Vert }^2\\ =\frac{\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2+\beta \langle A{x}^{k+1}+y^{k+1}-b,y^k-y^{k+1}\rangle \end{array}$ (3.9)

Subsequently, we proceed to estimate the remaining terms.

$\begin{array}{l}\left({\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;1,\cdot \right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;\alpha ,y^k\right)\right)\\ +\left({\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^k;\alpha ,y^k\right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^{k+1};1,\cdot \right)\right)\end{array}$

Indeed,

$\begin{array}{l}{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;1,\cdot \right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;\alpha ,y^k\right)\\ =f\left(x^{k+1}\right)+g\left(y^k\right)-\langle {\lambda }^k,A{x}^{k+1}+y^k-b\rangle +\beta D_h\left(-y^k,A{x}^{k+1}-b\right)\\ -\left[f\left(x^{k+1}\right)+g\left(y^k\right)-\langle {\lambda }^k,A{x}^{k+1}+y^k-b\rangle +\beta D_h\left(-\alpha y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\right]\\ =\beta D_h\left(-y^k,A{x}^{k+1}-b\right)-\beta D_h\left(-\alpha y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\end{array}$

and

$\begin{array}{l}{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^k;\alpha ,y^k\right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^{k+1};1,\cdot \right)\\ =f\left(x^{k+1}\right)+g\left(y^{k+1}\right)-\langle {\lambda }^k,A{x}^{k+1}+y^{k+1}-b\rangle +\beta D_h\left(-y^{k+1}-\left(\alpha -1\right)y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\\ -\left[f\left(x^{k+1}\right)+g\left(y^{k+1}\right)-\langle {\lambda }^{k+1},A{x}^{k+1}+y^{k+1}-b\rangle +\beta D_h\left(-y^{k+1},A{x}^{k+1}-b\right)\right]\\ =\langle {\lambda }^{k+1}-{\lambda }^k,A{x}^{k+1}+y^{k+1}-b\rangle +\beta D_h\left(-y^{k+1}-\left(\alpha -1\right)y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\\ -\beta D_h\left(-y^{k+1},A{x}^{k+1}-b\right)\end{array}$

By merging the two equalities, we obtain:

$\begin{array}{l}\left({\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;1,\cdot \right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;\alpha ,y^k\right)\right)\\ +\left({\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^k;\alpha ,y^k\right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^{k+1};1,\cdot \right)\right)\\ =\langle {\lambda }^{k+1}-{\lambda }^k,A{x}^{k+1}+y^{k+1}-b\rangle +\beta D_h\left(-y^k,A{x}^{k+1}-b\right)-\beta D_h\left(-y^{k+1},A{x}^{k+1}-b\right)\\ +\beta D_h\left(-y^{k+1}-\left(\alpha -1\right)y^k,\alpha \left(A{x}^{k+1}-b\right)\right)-\beta D_h\left(-\alpha y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\\ =\beta D_h\left(-y^k,-y^{k+1}\right)-\beta D_h\left(-\alpha y^k,-y^{k+1}-\left(\alpha -1\right)y^k\right)\\ +\langle {\lambda }^{k+1}-{\lambda }^k,A{x}^{k+1}+y^k-b\rangle +\beta \langle \nabla h\left(A{x}^{k+1}-b\right)-\nabla h\left(-y^{k+1}\right),y^k-y^{k+1}\rangle \end{array}$ (3.10)

The final equation is derived by employing the three-point equation (3.11) (3.121) and incorporating the optimality conditions (3.2c).

$\begin{array}{l}\beta D_h\left(-y^k-\left(\alpha -1\right)y^k,\alpha \left(A{x}^{k+1}-b\right)\right)-\beta D_h\left(-y^{k+1}-\left(\alpha -1\right)y^k,\alpha \left(A{x}^{k+1}-b\right)\right)\\ =\beta D_h\left(-\alpha y^k,-y^{k+1}-\left(\alpha -1\right)y^k\right)\\ +\beta \langle \nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right),y^k-y^{k+1}\rangle \\ =\beta D_h\left(-\alpha y^k,-y^{k+1}-\left(\alpha -1\right)y^k\right)+\langle {\lambda }^k-{\lambda }^{k+1},y^k-y^{k+1}\rangle \end{array}$ (3.11)

$\begin{array}{l}\beta D_h\left(-y^{k+1},A{x}^{k+1}-b\right)-\beta D_h\left(-y^k,A{x}^{k+1}-b\right)\\ =\beta D_h\left(-y^{k+1},-y^k\right)+\beta \langle \nabla h\left(A{x}^{k+1}-b\right)-\nabla h\left(-y^k\right),y^{k+1}-y^k\rangle \\ =-\beta D_h\left(-y^k,-y^{k+1}\right)-\beta \langle \nabla h\left(A{x}^{k+1}-b\right)-\nabla h\left(-y^{k+1}\right),y^k-y^{k+1}\rangle \end{array}$ (3.12)

Given that $x^{k+1}$ is the minimizer of ${ℒ}_{\beta }^h\left(x,y^k,{\lambda }^k\right)$ with respect to the variable $x$, we have:

${\widehat{ℒ}}_{\beta }^h\left(x^k,y^k,{\lambda }^k;1,\cdot \right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;1,\cdot \right)={ℒ}_{\beta }^h\left(x^k,y^k,{\lambda }^k\right)-{ℒ}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k\right)\ge 0$ (3.13)

Obsevre that

$\begin{array}{c}{ℒ}_{\beta }^h\left({\omega }^k\right)-{ℒ}_{\beta }^h\left({\omega }^{k+1}\right)={ℒ}_{\beta }^h\left(x^k,y^k,{\lambda }^k\right)-{ℒ}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^{k+1}\right)\\ ={\widehat{ℒ}}_{\beta }^h\left(x^k,y^k,{\lambda }^k;1,\cdot \right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^{k+1};1,\cdot \right)\\ =\left({\widehat{ℒ}}_{\beta }^h\left(x^k,y^k,{\lambda }^k;1,\cdot \right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;1,\cdot \right)\right)\\ +({\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;1,\cdot \right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;\alpha ,y^k\right))\\ +\left({\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k;\alpha ,y^k\right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^k;\alpha ,y^k\right)\right)\\ +\left({\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^k;\alpha ,y^k\right)-{\widehat{ℒ}}_{\beta }^h\left(x^{k+1},y^{k+1},{\lambda }^{k+1};1,\cdot \right)\right)\end{array}$

Consequently, by adding up the inequalities (3.8), (3.10) and (3.13), we arrive at the conclusion that:

$\begin{array}{l}{ℒ}_{\beta }^h\left({\omega }^k\right)-{ℒ}_{\beta }^h\left({\omega }^{k+1}\right)\\ \ge -L{D}_h\left(y^k,y^{k+1}\right)+\langle {\lambda }^{k+1}-{\lambda }^k,A{x}^{k+1}+y^k-b\rangle \\ +\frac{\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2+\beta \langle A{x}^{k+1}+y^{k+1}-b,y^k-y^{k+1}\rangle \\ +\beta D_h\left(-y^k,-y^{k+1}\right)+\beta \langle \nabla h\left(A{x}^{k+1}-b\right)-\nabla h\left(-y^{k+1}\right),y^k-y^{k+1}\rangle \\ \ge -L{D}_h\left(y^k,y^{k+1}\right)-\Vert {\lambda }^{k+1}-{\lambda }^k\Vert \cdot \Vert A{x}^{k+1}+y^k-b\Vert +\frac{\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2\\ +\beta D_h\left(-y^k,-y^{k+1}\right)-\beta \cdot \left(L_h+1\right)\Vert A{x}^{k+1}+y^{k+1}-b\Vert \cdot \Vert y^{k+1}-y^{k+1}\Vert \end{array}$ (3.14)

Considering the 1-strong convexity and $L_h$-smoothness of the function $h(\cdot )$, we can respectively derive the following inequalities:

$\begin{array}{c}\frac{1}{2}{\Vert y^k-y^{k+1}\Vert }^2\le D_h\left(y^k,y^{k+1}\right)=h\left(y^k\right)-h\left(y^{k+1}\right)-\langle \nabla h\left(y^{k+1}\right),y^k-y^{k+1}\rangle \\ \le \frac{L_h}{2}{\Vert y^k-y^{k+1}\Vert }^2\end{array}$

$\frac{1}{2}{\Vert y^k-y^{k+1}\Vert }^2\le h\left(-y^k\right)-h\left(-y^{k+1}\right)-\langle \nabla h\left(-y^{k+1}\right),-y^k+y^{k+1}\rangle =D_h\left(-y^k,-y^{k+1}\right)$

$\begin{array}{c}{D}_h\left(-y^k,-y^{k+1}\right)=h\left(-y^k\right)-h\left(-y^{k+1}\right)-\langle \nabla h\left(-y^{k+1}\right),-y^k+y^{k+1}\rangle \\ \ge \frac{1}{2}{\Vert y^k-y^{k+1}\Vert }^2\ge \frac{1}{L_h}{D}_h\left(y^k,y^{k+1}\right)\end{array}$

Given that $h$ is 1-strong convex, we can deduce that:

$\begin{array}{l}\Vert \nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\Vert \cdot \Vert \alpha \left(A{x}^{k+1}-b\right)-\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\Vert \\ \ge \langle \nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right),\alpha \left(A{x}^{k+1}-b\right)-\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\rangle \\ \ge {\Vert \alpha \left(A{x}^{k+1}-b\right)-\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\Vert }^2\end{array}$ (3.16)

Hence, from the aforementioned equation (3.15), we are able to derive the following:

$\begin{array}{l}\Vert \nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\Vert \\ \ge \Vert \alpha \left(A{x}^{k+1}-b\right)-\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\Vert \\ =\Vert \alpha \left(A{x}^{k+1}+y^k-b\right)+\left(y^{k+1}-y^k\right)\Vert \end{array}$ (3.16)

Combining optimimal condition (3.2c)

$\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)=\frac{{\lambda }^k-{\lambda }^{k+1}}{\beta }$ (3.17)

Additionally, by applying the triangle inequality, we can know that:

$\Vert \alpha \left(A{x}^{k+1}+y^k-b\right)\Vert \le \Vert y^k-y^{k+1}\Vert +\Vert \alpha \left(A{x}^{k+1}+y^k-b\right)+\left(y^{k+1}-y^k\right)\Vert$ (3.18)

So combining (3.6), (3.17) and (3.18) we can derive the following result

$\begin{array}{c}\Vert A{x}^{k+1}+y^k-b\Vert \le \frac{\Vert y^k-y^{k+1}\Vert +\Vert \alpha \left(A{x}^{k+1}+y^k-b\right)+\left(y^{k+1}-y^k\right)\Vert }{\alpha }\\ \le \frac{\Vert y^k-y^{k+1}\Vert }{\alpha }+\frac{\Vert {\lambda }^k-{\lambda }^{k+1}\Vert }{\alpha \cdot \beta }\end{array}$ (3.19)

And by the same token, we can get an inequality for $A{x}^{k+1}+y^{k+1}-b$

$\begin{array}{l}\Vert A{x}^{k+1}+y^{k+1}-b\Vert \\ \le \frac{\Vert \left(1-\alpha \right)\left(y^k-y^{k+1}\right)\Vert +\Vert \alpha \left(A{x}^{k+1}+y^k-b\right)+\left(y^{k+1}-y^k\right)\Vert }{\alpha }\\ \le \frac{\Vert \left(1-\alpha \right)\left(y^k-y^{k+1}\right)\Vert }{\alpha }+\frac{\Vert {\lambda }^k-{\lambda }^{k+1}\Vert }{\alpha \cdot \beta }\end{array}$ (3.20)

Next, we declare that

$\Vert \nabla g\left(y^{k+1}\right)-\nabla g\left(y^k\right)\Vert \le L{L}_h\cdot \Vert y^{k+1}-y^k\Vert .$ (3.21)

To prove (3.21), we consider two cases. When $y^k=y^{k+1}$, (3.21) holds trivially. Now, we assume $y^{k+1}\ne y^k$. Since $\nabla g(\cdot )$ is $D$-Lipschitz, we have

$\begin{array}{c}\Vert \nabla g\left(y^{k+1}\right)-\nabla g\left(y^k\right)\Vert \le L\cdot \frac{D_h\left(y^{k+1},y^k\right)+D_h\left(y^k,y^{k+1}\right)}{\Vert y^{k+1}-y^k\Vert }\\ =L\cdot \frac{\langle \nabla h\left(y^k\right)-\nabla h\left(y^{k+1}\right),y^k-y^{k+1}\rangle }{\Vert y^{k+1}-y^k\Vert }\end{array}$

$\begin{array}{c}\le L\cdot \frac{\Vert \nabla h\left(y^k\right)-\nabla h\left(y^{k+1}\right)\Vert \cdot \Vert y^k-y^{k+1}\Vert }{\Vert y^{k+1}-y^k\Vert }\\ \le L\cdot \frac{L_h{\Vert y^k-y^{k+1}\Vert }^2}{\Vert y^{k+1}-y^k\Vert }=L{L}_h\cdot \Vert y^{k+1}-y^k\Vert ,\end{array}$

where the second inequality follows from that $\nabla h$ is Lipschitz continuous with $L_h\left(L_h\ge 1\right)$ on any bounded subset of ${ℝ}^m$, that is

$\Vert \nabla h\left(y^{k+1}\right)-\nabla h\left(y^k\right)\Vert \le L_h\Vert y^{k+1}-y^k\Vert .$

Since ${\lambda }^{k+1}=\nabla g\left(y^{k+1}\right)$, (3.21) becomes

$\Vert {\lambda }^{k+1}-{\lambda }^k\Vert \le L{L}_h\Vert y^{k+1}-y^k\Vert .$ (3.22)

And we also know

${\Vert {\lambda }^k-{\lambda }^{k+1}\Vert }^2={\Vert \nabla g\left(y^k\right)-\nabla g\left(y^{k+1}\right)\Vert }^2\le L^2{L}_h^2{\Vert y^{k+1}-y^k\Vert }^2$ (3.23)

And

$\Vert {\lambda }^k-{\lambda }^{k+1}\Vert \Vert y^{k+1}-y^k\Vert =\Vert \nabla g\left(y^k\right)-\nabla g\left(y^{k+1}\right)\Vert \Vert y^{k+1}-y^k\Vert \le L{L}_h{\Vert y^{k+1}-y^k\Vert }^2$

Since $\alpha >1$ and $\alpha <1$ will affect the result of our calculation, we will classify and discuss them based on the different values of $\alpha$.

i) if $\alpha \in \left(\frac{L_h+L_h^2}{1+L_h+L_h^2},1\right]$, then we put (3.19) (3.20) and (3.22) into (3.14), we can get

$\begin{array}{l}{ℒ}_{\beta }^h\left({\omega }^k\right)-{ℒ}_{\beta }^h\left({\omega }^{k+1}\right)\\ \ge -L{D}_h\left(y^k,y^{k+1}\right)-\frac{\Vert y^k-y^{k+1}\Vert \Vert {\lambda }^k-{\lambda }^{k+1}\Vert }{\alpha }-\frac{{\Vert {\lambda }^k-{\lambda }^{k+1}\Vert }^2}{\alpha \beta }+\frac{\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2\\ +\frac{\beta }{L_h}{D}_h\left(y^k,y^{k+1}\right)-\beta \left(L_h+1\right)\left[\frac{\Vert \left(1-\alpha \right)\left(y^k-y^{k+1}\right)\Vert \Vert y^k-y^{k+1}\Vert }{\alpha }+\frac{\Vert {\lambda }^k-{\lambda }^{k+1}\Vert \Vert y^k-y^{k+1}\Vert }{\alpha \beta }\right]\\ \ge -L{D}_h\left(y^k,y^{k+1}\right)+\frac{\beta }{L_h}{D}_h\left(y^k,y^{k+1}\right)+\frac{\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2-\frac{2L{L}_h}{\alpha }{\Vert y^k-y^{k+1}\Vert }^2\\ -\frac{L^2{L}_h^2{\Vert y^k-y^{k+1}\Vert }^2}{\alpha \beta }-\beta \frac{\left(1+L_h\right)\left(1-\alpha \right){\Vert y^k-y^{k+1}\Vert }^2}{\alpha }-\frac{L{L}_h^2{\Vert y^k-y^{k+1}\Vert }^2}{\alpha }\\ \ge \left(\frac{2\beta }{L_h}-L\right)D_h\left(y^k,y^{k+1}\right)+\left[-\frac{\beta \cdot \left(1-\alpha \right)\left(1+L_h\right)}{\alpha }-\frac{2L{L}_h}{\alpha }-\frac{L{L}_h^2}{\alpha }-\frac{L^2\cdot L_h^2}{\alpha \beta }\right]{\Vert y^k-y^{k+1}\Vert }^2\\ \ge \left[\frac{2\beta }{L_h}-\frac{2\beta \left(1-\alpha \right)\left(1+L_h\right)}{\alpha }-\frac{\alpha L+4L{L}_h+2L{L}_h^2}{\alpha }-\frac{2L^2{L}_h^2}{\alpha \beta }\right]D_h\left(y^k,y^{k+1}\right)\\ \ge \left[\frac{2\beta }{L_h}-\frac{2\beta \left(1-\alpha \right)\left(1+L_h\right)}{\alpha }-\frac{7L{L}_h^2}{\alpha }-\frac{2L^2{L}_h^2}{\alpha \beta }\right]D_h\left(y^k,y^{k+1}\right)\end{array}$ (3.24)

ii) if $\alpha \in \left[1,\frac{-L_h-L_h^2}{1-L_h-L_h^2}\right)$, and from the formula above we can get

$\begin{array}{l}{ℒ}_{\beta }^h\left({\omega }^k\right)-{ℒ}_{\beta }^h\left({\omega }^{k+1}\right)\\ \ge -L{D}_h\left(y^k,y^{k+1}\right)-\frac{\Vert y^k-y^{k+1}\Vert \Vert {\lambda }^k-{\lambda }^{k+1}\Vert }{\alpha }-\frac{{\Vert {\lambda }^k-{\lambda }^{k+1}\Vert }^2}{\alpha \beta }+\frac{\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2\\ +\frac{\beta }{L_h}{D}_h\left(y^k,y^{k+1}\right)-\beta \left(L_h+1\right)\left[\frac{\Vert \left(1-\alpha \right)\left(y^k-y^{k+1}\right)\Vert \Vert y^k-y^{k+1}\Vert }{\alpha }+\frac{\Vert {\lambda }^k-{\lambda }^{k+1}\Vert \Vert y^k-y^{k+1}\Vert }{\alpha \beta }\right]\\ \ge -L{D}_h\left(y^k,y^{k+1}\right)+\frac{\beta }{L_h}{D}_h\left(y^k,y^{k+1}\right)+\frac{\beta }{2}{\Vert y^k-y^{k+1}\Vert }^2-\frac{2L{L}_h}{\alpha }{\Vert y^k-y^{k+1}\Vert }^2\\ -\frac{L^2{L}_h^2{\Vert y^k-y^{k+1}\Vert }^2}{\alpha \beta }-\beta \frac{\left(1+L_h\right)\left(\alpha -1\right){\Vert y^k-y^{k+1}\Vert }^2}{\alpha }-\frac{L{L}_h^2{\Vert y^k-y^{k+1}\Vert }^2}{\alpha }\\ \ge \left(\frac{2\beta }{L_h}-L\right)D_h\left(y^k,y^{k+1}\right)+\left[-\frac{\beta \cdot \left(\alpha -1\right)\left(1+L_h\right)}{\alpha }-\frac{2L{L}_h}{\alpha }-\frac{L{L}_h^2}{\alpha }-\frac{L^2\cdot L_h^2}{\alpha \beta }\right]{\Vert y^k-y^{k+1}\Vert }^2\\ \ge \left[\frac{2\beta }{L_h}-\frac{2\beta \left(\alpha -1\right)\left(1+L_h\right)}{\alpha }-\frac{\alpha L+4L{L}_h+2L{L}_h^2}{\alpha }-\frac{2L^2{L}_h^2}{\alpha \beta }\right]D_h\left(y^k,y^{k+1}\right)\\ \ge \left[\frac{2\beta }{L_h}-\frac{2\beta \left(\alpha -1\right)\left(1+L_h\right)}{\alpha }-\frac{\left(\alpha +6\right)L{L}_h^2}{\alpha }-\frac{2L^2{L}_h^2}{\alpha \beta }\right]D_h\left(y^k,y^{k+1}\right)\end{array}$ (3.25)

The proof is complete.

Lemma 3.2 Let $\left\{w^k=\left(x^k,y^k,{\lambda }^k\right)\right\}$ be the sequence generated by the Generalized Bregman ADMM (1.6), which is assumed to be bounded. Then we have

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}{\Vert w^{k+1}-w^k\Vert }^2<+\infty .$ (3.26)

Proof: Given that the sequence $\left\{w^k\right\}$ is bounded, it follows that there exists a subsequence $\left\{w^{k_j}\right\}$ such that $w^{k_j}\to w^{\text{*}}$. As $f(\cdot )$ is lower semicontinuous and $g(\cdot )$ is continuous, it can be deduced that the function ${ℒ}_{\beta }^h(\cdot )$ is also lower semicontinuous. Therefore,

${ℒ}_{\beta }^h\left(w^{*}\right)\le \underset{j\to +\infty }{\lim \inf }{ℒ}_{\beta }^h\left(w^{k_j}\right).$

As a result, $\left\{{ℒ}_{\beta }^h\left(w^{k_j}\right)\right\}$is bounded from below. Additionally, since $\left\{{ℒ}_{\beta }^h\left(w^k\right)\right\}$ is nonincreasing, it follows that $\left\{{ℒ}_{\beta }^h\left(w^{k_j}\right)\right\}$ is convergent. Moreover, $\left\{{ℒ}_{\beta }^h\left(w^k\right)\right\}$ is convergent, and ${ℒ}_{\beta }^h\left(w^k\right)\ge {ℒ}_{\beta }^h\left(w^{\text{*}}\right)$. According to equation (3.5), we have

$\delta \cdot D_h\left(y^k,y^{k+1}\right)\le {ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{k+1}\right).$

Summing over $k=0,\cdots ,n$, it follows

$\underset{k=0}{\overset{n}{{\displaystyle \sum }}}\delta D_h\left(y^k,y^{k+1}\right)\le {ℒ}_{\beta }^h\left(w^0\right)-{ℒ}_{\beta }^h\left(w^{n+1}\right)\le {ℒ}_{\beta }^h\left(w^0\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)<+\infty .$

Since $\delta >0$, we have $\underset{k=0}{\overset{\infty }{{\displaystyle \sum }}}{D}_h\left(y^k,y^{k+1}\right)<+\infty$, which implies

$\underset{k=0}{\overset{\infty }{{\displaystyle \sum }}}{\Vert y^{k+1}-y^k\Vert }^2<+\infty$. Hence, it follows from (3.22) that $\underset{k=0}{\overset{\infty }{{\displaystyle \sum }}}{\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2<+\infty$.

Recall that

${\lambda }^k={\lambda }^{k-1}-\beta \left(\nabla h\left(\alpha \left(A{x}^k-b\right)\right)-\nabla h\left(-y^k-\left(\alpha -1\right)y^{k-1}\right)\right),$

${\lambda }^{k+1}={\lambda }^k-\beta \left(\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)\right)$

Subtracting the first equality from the second equality, we obtain

$\begin{array}{c}{\lambda }^{k+1}-{\lambda }^k={\lambda }^k-{\lambda }^{k-1}+\beta \left(\nabla h\left(\alpha \left(A{x}^k-b\right)\right)-\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)\right)\\ +\beta \left(\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)-\nabla h\left(-y^k-\left(\alpha -1\right)y^{k-1}\right)\right).\end{array}$

Rearranging the above equation and taking the square of the ${\ell }_2$-norm, it follows

$\begin{array}{l}{\Vert \beta \left(\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(\alpha \left(A{x}^k-b\right)\right)\right)\Vert }^2\\ ={\Vert {\lambda }^{k+1}-{\lambda }^k-\left({\lambda }^k-{\lambda }^{k-1}\right)-\beta \left(\nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)-\nabla h\left(-y^k-\left(\alpha -1\right)y^{k-1}\right)\right)\Vert }^2\\ \le 3\left({\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2+{\Vert {\lambda }^k-{\lambda }^{k-1}\Vert }^2+{\beta }^2{\Vert \nabla h\left(-y^{k+1}-\left(\alpha -1\right)y^k\right)-\nabla h\left(-y^k-\left(\alpha -1\right)y^{k-1}\right)\Vert }^2\right)\\ \le 3\left({\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2+{\Vert {\lambda }^k-{\lambda }^{k-1}\Vert }^2+{\beta }^2{L}_h^2{\Vert y^{k+1}-y^k+\left(\alpha -1\right)\left(y^k-y^{k+1}\right)\Vert }^2\right)\\ \le 3({\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2+{\Vert {\lambda }^k-{\lambda }^{k-1}\Vert }^2+2{\beta }^2{L}_h^2{\Vert y^{k+1}-y^k\Vert }^2+2\left(\alpha -1{)}^2{\beta }^2{L}_h^2{\Vert y^k-y^{k+1}\Vert }^2\right).\end{array}$ (3.27)

On the other hand,

${\beta }^2{\alpha }^2{\Vert A{x}^{k+1}-A{x}^k\Vert }^2\le {\Vert \beta \left(\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(\alpha \left(A{x}^k-b\right)\right)\right)\Vert }^2.$ (3.28)

From (ii) of Assumption 3.1, we note that

${\Vert A{x}^{k+1}-A{x}^k\Vert }^2\ge M{\Vert x^{k+1}-x^k\Vert }^2.$ (3.29)

Combining (3.27) - (3.29) together, we get

$\begin{array}{l}{\alpha }^2{\beta }^{2}M{\Vert x^{k+1}-x^k\Vert }^2\le {\alpha }^2{\beta }^2{\Vert A{x}^{k+1}-A{x}^k\Vert }^2\\ \le {\Vert \beta \left(\nabla h\left(\alpha \left(A{x}^{k+1}-b\right)\right)-\nabla h\left(\alpha \left(A{x}^k-b\right)\right)\right)\Vert }^2\\ \le 3({\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2+{\Vert {\lambda }^k-{\lambda }^{k-1}\Vert }^2+2{\beta }^2{L}_h^2{\Vert y^{k+1}-y^k\Vert }^2\\ +2{\left(\alpha -1\right)}^2{\beta }^2{L}_h^2{\Vert y^k-y^{k+1}\Vert }^2)\end{array}$ (3.30)

where $M>0$. Then, (3.30) implies ${\displaystyle {\sum }_{k=0}^{+\infty }{\Vert x^{k+1}-x^k\Vert }^2}<+\infty$. Thus, ${\displaystyle {\sum }_{k=0}^{+\infty }{\Vert w^{k+1}-w^k\Vert }^2}<+\infty$. This completes the proof. □

Lemma 3.3 t ${\left\{w^k=\left(x^k,y^k,{\lambda }^k\right)\right\}}_k\in N$ be the sequence generated by the Generalized Bregman ADMM (1.3), which is assumed to be bounded. Furthermore, there exists $\eta >0$ such that

$d\left(0,\partial {ℒ}_{\beta }^h\left(w^{k+1}\right)\right)\le \eta \Vert y^{k+1}-y^k\Vert .$

Proof: By definition of function ${ℒ}_{\beta }^h(\cdot )$, we have the following system of equations:

$\{\begin{array}{l}{\partial }_x{ℒ}_{\beta }^h\left({\omega }^{k+1}\right)=\partial f\left(x^{k+1}\right)-A^{\text{T}}{\lambda }^{k+1}+\beta A^{\top }\left(A{x}^{k+1}+y^{k+1}-b\right)+\beta A^{\text{T}}\langle {\nabla }^{2}h\left(A{x}^{k+1}-b\right),A{x}^{k+1}+y^{k+1}-b\rangle ,\\ {\partial }_y{ℒ}_{\beta }^h\left({\omega }^{k+1}\right)=\nabla g\left(y^{k+1}\right)-{\lambda }^{k+1}+\beta \left(A{x}^{k+1}+y^{k+1}-b\right)+\beta \left(\nabla h\left(A{x}^{k+1}-b\right)-\nabla h\left(-y^{k+1}\right)\right),\\ {\partial }_{\lambda }{ℒ}_{\beta }^h\left({\omega }^{k+1}\right)=-\left(A{x}^{k+1}+y^{k+1}-b\right).\end{array}$ (3.31)

Combining equation (3.31) with optimality condition (3.1), we obtain:

$\{\begin{array}{l}{A}^{\text{T}}\left({\lambda }^k-{\lambda }^{k+1}\right)+\beta A^{\top }\left(y^{k+1}-y^k\right)+\beta A^{\text{T}}\langle {\nabla }^{2}h\left(A{x}^{k+1}-b\right),A{x}^{k+1}+y^{k+1}-b\rangle \in {\partial }_x{ℒ}_{\beta }^h\left(w^{k+1}\right),\\ \beta \left(A{x}^{k+1}+y^{k+1}-b\right)+\beta \left(\nabla h\left(A{x}^{k+1}-b\right)-\nabla h\left(-y^{k+1}\right)\right)\in {\partial }_y{ℒ}_{\beta }^h\left(w^{k+1}\right),\\ -\left(A{x}^{k+1}+y^{k+1}-b\right)\in {\partial }_{\lambda }{ℒ}_{\beta }^h\left(w^{k+1}\right).\end{array}$

In addition, By the $L_h$-smoothness of the function $h(\cdot )$, we can deduce that

$\Vert \nabla h\left(A{x}^{k+1}-b\right)-\nabla h\left(-y^{k+1}\right)\Vert \le L_h\cdot \Vert A{x}^{k+1}+y^{k+1}-b\Vert$

From formula (3.20) we know the following result

$\Vert A{x}^{k+1}+y^{k+1}-b\Vert \le \frac{\Vert \left(1-\alpha \right)\left(y^k-y^{k+1}\right)\Vert }{\alpha }+\frac{\Vert {\lambda }^k-{\lambda }^{k+1}\Vert }{\alpha \cdot \beta }$

In addition,

$\begin{array}{l}\langle \nabla h^2\left(A{x}^{k+1}-b\right),A{x}^{k+1}+y^{k+1}-b\rangle \\ \le \Vert \nabla h^2\left(A{x}^{k+1}-b\right)\Vert \Vert A{x}^{k+1}+y^{k+1}-b\Vert \le L_h\Vert A{x}^{k+1}+y^{k+1}-b\Vert .\end{array}$

Thus, if we set

$\begin{array}{l}\left({\xi }_1^{k+1},{\xi }_2^{k+1},{\xi }_3^{k+1}\right)\\ :=(A^{\text{T}}\left({\lambda }^k-{\lambda }^{k+1}\right)+\beta A^{\top }\left(y^{k+1}-y^k\right)+\beta A^{\text{T}}\langle {\nabla }^{2}h\left(A{x}^{k+1}-b\right),A{x}^{k+1}+y^{k+1}-b\rangle ,\\ \beta \left(A{x}^{k+1}+y^{k+1}-b\right)+\beta \left(\nabla h\left(A{x}^{k+1}-b\right)-\nabla h\left(-y^{k+1}\right)\right),-\left(A{x}^{k+1}+y^{k+1}-b\right))\end{array}$

Then it follows from lemma 2.3 that $\left({\xi }_1^{k+1},{\xi }_2^{k+1},{\xi }_3^{k+1}\right)\in \partial ℒ\beta \left({\omega }^{k+1}\right)$. Moreover, there exist ${\eta }_1,{\eta }_2>0$ such that

$\Vert \left({\xi }_1^{k+1},{\xi }_2^{k+1},{\xi }_3^{k+1}\right)\Vert \le {\eta }_1\Vert y^{k+1}-y^k\Vert +{\eta }_2\Vert {\lambda }^{k+1}-{\lambda }^k\Vert$

Notice that, we can deduce from (3.22) that

$\Vert {\lambda }^{k+1}-{\lambda }^k\Vert \le L\cdot L_n\Vert y^{k+1}-y^k\Vert$

We define $\eta :={\eta }_1+{\eta }_{2}L{L}_n$, it follows from above... that

$\begin{array}{c}d\left(0,\partial {ℒ}_{\beta }^h\left(w^{k+1}\right)\right)\le \Vert \left({\xi }_1^{k+1},{\xi }_2^{k+1},{\xi }_3^{k+1}\right)\Vert \le {\eta }_1\Vert y^{k+1}-y^k\Vert +{\eta }_2\Vert {\lambda }^{k+1}-{\lambda }^k\Vert \\ \le \left({\eta }_1+{\eta }_{2}L{L}_n\right)\cdot \Vert y^{k+1}-y^k\Vert \\ \le \eta \Vert y^{k+1}-y^k\Vert \end{array}$

This completes the proof. □

Lemma 3.4 Let $\left\{w^k=\left(x^k,y^k,{\lambda }^k\right)\right\}$ be the sequence generated by the Generalized Bregman ADMM (1.3), which is assumed to be bounded. Let $S\left(w^0\right)$ denote the set of its limit points. Then

i) $S\left(w_0\right)$ is a nonempty compact set, and

$d\left(w^k,S\left(w^0\right)\right)\to 0,\text{as}k\to +\infty ;$

ii) $S\left(w^0\right)\subset crit{ℒ}_{\beta }^h$, where $crit{ℒ}_{\beta }^h$ denotes the set of all stationary points of ${ℒ}_{\beta }^h$;

iii) ${ℒ}_{\beta }^h(\cdot )$ is finite and constant on $S\left(w_0\right)$, which equals to

$\underset{k\in \mathbb{N}}{\text{inf}}{ℒ}_{\beta }^h\left(w^k\right)=\underset{k\to +\infty }{\text{lim}}{ℒ}_{\beta }^h\left(w^k\right).$

Proof: We proof the results item by item.

i) The item follows as an elementary consequence of the definition of limit points.

ii) For any fixed $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\in S\left(w_0\right)$, then there exists a subsequence ${\left\{\left(x^{k_j},y^{k_j},{\lambda }^{k_j}\right)\right\}}_j\in N$ that converges to $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$. By the definition of the augmented Lagrangian function (3.3), the x-subproblem of (1.6) is equivalent to

$x^{k+1}\in \text{arg}\underset{x}{\text{min}}\left\{{ℒ}_{\beta }^h\left(x,y^k,{\lambda }^k\right)\right\}$

that means $x^{k+1}$ is the global minimizer of ${ℒ}_{\beta }^h\left(x,y^k,{\lambda }^k\right)$ for the variable $x$, then it holds that

${ℒ}_{\beta }^h\left(x^{k+1},y^k,{\lambda }^k\right)\le {ℒ}_{\beta }^h\left(x^{\text{*}},y^k,{\lambda }^k\right).$ (3.32)

On one hand, using (3.32) and the continuity of ${ℒ}_{\beta }^h(\cdot )$ with respect to $y$ and $\lambda$, we have

$\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^h\left(x^{k_j+1},y^{k_j},{\lambda }^{k_j}\right)=\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^h\left(x^{k_j+1},y^{k_j+1},{\lambda }^{k_j+1}\right)\le {ℒ}_{\beta }^h\left(x^{*},y^{*},{\lambda }^{*}\right).$ (3.33)

On the other hand, (3.26) implies $\Vert w^{k+1}-w^k\Vert \to 0$, which means that the subsequence

${\left\{\left(x^{k_j+1},y^{k_j+1},{\lambda }^{k_j+1}\right)\right\}}_{j\in N}$

also converges to $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$. From the lower semicontinuity of ${ℒ}_{\beta }^h(\cdot )$, we have

$\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^h\left(x^{k_j+1},y^{k_j},{\lambda }^{k_j}\right)\ge {ℒ}_{\beta }^h\left(x^{*},y^{*},{\lambda }^{*}\right).$ (3.34)

Then by combining (3.33) and (3.34) together we can get

$\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }^h\left(x^{k_j+1},y^{k_j},{\lambda }^{k_j}\right)={ℒ}_{\beta }^h\left(x^{*},y^{*},{\lambda }^{*}\right).$

which implies

$\underset{j\to +\infty }{\text{lim}}f\left(x^{k_j+1}\right)=f\left(x^{\text{*}}\right)$ (3.35)

Passing to the limit in (3.2) along the subsequence ${\left\{\left(x^{k_j+1},y^{k_j+1},{\lambda }^{k_j+1}\right)\right\}}_{j\in N}$

and invoking (3.35) and the continuity of $\nabla g$, it follows that

$\{\begin{array}{l}{A}^{\top }{\lambda }^{\text{*}}\in \partial f\left(x^{\text{*}}\right),\\ {\lambda }^{\text{*}}=\nabla g\left(y^{\text{*}}\right),\\ \nabla h\left(\alpha \left(A{x}^{\text{*}}-b\right)\right)-\nabla h\left(-\alpha y^{\text{*}}\right)=0.\end{array}$

The last equation implies that $A{x}^{\text{*}}+y^{\text{*}}=b$ due to the strong convexity of $h(\cdot )$. Thus, $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$ is a critical point of (3.3), which implies that $w^{\text{*}}\in \text{crit}{ℒ}_{\beta }^h$.

iii) For any point $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\in S\left(w_0\right)$, there exists a subsequence $\left\{\left(x^{k_j},y^{k_j},{\lambda }^{k_j}\right)\right\}$ that converges to $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$. Combining equations (3.33), (3.34) and the fact that $\left\{{ℒ}_{\beta }^h\left(w^k\right)\right\}$ is nonincreasing, we can get

$\underset{k\to +\infty }{\text{lim}}{ℒ}_{\beta }^h\left(x^k,y^k,{\lambda }^k\right)={ℒ}_{\beta }^h\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right).$

Therefore, ${ℒ}_{\beta }^h(\cdot )$ is finite and constant on $S\left(w_0\right)$, Moreover,

$\underset{k\in \mathbb{N}}{\text{inf}}{ℒ}_{\beta }^h\left(w^k\right)=\underset{k\to +\infty }{\text{lim}}{ℒ}_{\beta }^h\left(w^k\right)$

The proof is completed. □

In the following, we will present an important result of this paper, which provides a detailed analysis of the convergence of Generalized Bregman ADMM (1.4).

Theorem 3.1 Let $\left\{w^k=\left(x^k,y^k,{\lambda }^k\right)\right\}$ be the sequence generated by the Generalized Bregman ADMM (1.3), which is assumed to be bounded. Suppose that ${ℒ}_{\beta }^h(\cdot )$ is a KL function, then $\left\{w^k\right\}$ has finite length, that is

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}\Vert w^{k+1}-w^k\Vert <+\infty ,$

and as a consequence, $\left\{w^k\right\}$ converges to a critical point of ${ℒ}_{\beta }^h(\cdot )$.

Proof: From the proof of Lemma 3.4, we know that ${ℒ}_{\beta }^h\left(w^k\right)\to {ℒ}_{\beta }^h\left(w^{\text{*}}\right)$ for all $w^{\text{*}}\in S\left(w_0\right)$. Let us now consider two cases.

i) If there exists an integer $k_0$ such that ${ℒ}_{\beta }^h\left(w^{k_0}\right)={ℒ}_{\beta }^h\left(w^{\text{*}}\right)$, then using (3.5), we have

$\delta D_h\left(y^k,y^{k+1}\right)\le {ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{k+1}\right)\le {ℒ}_{\beta }^h\left(w^{k_0}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)=0,$

for any $k>k_0$. Thus, we obtain $y^{k+1}=y^k$ for any $k>k_0$. Combining (3.22) and (3.30), we further derive that ${\lambda }^{k+1}={\lambda }^k$ and $x^{k+1}=x^k$ for any $k>k_0+1$, which implies that $w^{k+1}=w^k$. Hence, the assertion holds.

ii) If ${ℒ}_{\beta }^h\left(w^k\right)>{ℒ}_{\beta }^h\left(w^{\text{*}}\right)$ for all $k$, then since $d\left(w^k,S\left(w_0\right)\right)\to 0$, there exists $k_1>0$, such that for any $k>k_1$, we have $d\left(w^k,S\left(w_0\right)\right)<\epsilon$ for all $\epsilon >0$. Moreover, with ${ℒ}_{\beta }^h\left(w^k\right)\to {ℒ}_{\beta }^h\left(w^{\text{*}}\right)$, it follows that there exists $k_2>0$ such that for any $k>k_2$, ${ℒ}_{\beta }^h\left(w^k\right)<{ℒ}_{\beta }^h\left(w^{\text{*}}\right)+\eta$ for all $\eta >0$. Therefore, when $k>\tilde{k}=\text{max}\left\{k_1,k_2\right\}$ for all $\epsilon ,\eta >0$, we can obtain the following:

$d\left(w^k,S\left(w_0\right)\right)<\epsilon ,{ℒ}_{\beta }^h\left(w^{\text{*}}\right)<{ℒ}_{\beta }^h\left(w^k\right)<{ℒ}_{\beta }^h\left(w^{\text{*}}\right)+\eta .$

Since $S\left(w_0\right)$ is a nonempty compact set and ${ℒ}_{\beta }^h(\cdot )$ is constant on $S\left(w_0\right)$, we can apply Lemma 2.4 with $\Omega =S\left(w_0\right)$ to deduce that for any $k>\tilde{k}$,

$\phi \text{'}\left({ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)d\left(0,\partial {ℒ}_{\beta }^h\left(w^k\right)\right)\ge 1.$ (3.36)

Using the fact that ${ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{k+1}\right)={ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)-\left({ℒ}_{\beta }^h\left(w^{k+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)$, and the concavity of $\phi (\cdot )$, we can show that

$\begin{array}{l}\phi \left({ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^h\left(w^{k+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)\\ \ge {\phi }^{\prime }\left({ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)\left({ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{k+1}\right)\right).\end{array}$

Combining the above inequality with $d\left(0,\partial {ℒ}_{\beta }^h\left(w^k\right)\right)\le \xi \Vert y^k-y^{k-1}\Vert$, ${\phi }^{\prime }\left({ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)>0$ and relation (3.36), we obtain

$\begin{array}{l}{ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{k+1}\right)\le \frac{\phi \left({ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^h\left(w^{k+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)}{{\phi }^{\prime }\left({ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)}\\ \le d\left(0,\partial {ℒ}_{\beta }^h\left(w^k\right)\right)\left[\phi \left({ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^h\left(w^{k+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)\right]\\ \le \xi \Vert y^k-y^{k-1}\Vert \left[\phi \left({ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^h\left(w^{k+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)\right].\end{array}$ (3.37)

For convenience, we define ${\text{Δ}}_{p,q}:=\phi \left({ℒ}_{\beta }^h\left(w^p\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)-\phi \left({ℒ}_{\beta }^h\left(w^q\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)$. Then, (3.37) can be simplified as

${ℒ}_{\beta }^h\left(w^k\right)-{ℒ}_{\beta }^h\left(w^{k+1}\right)\le \xi \Vert y^k-y^{k-1}\Vert {\text{Δ}}_{k,k+1}.$ (3.38)

According to the 1-strong convexity of the function $h$, and combining Lemma 3.1 with inequality (3.38), we get that for all $k>\tilde{k}$,

$\frac{\delta }{2}{\Vert y^k-y^{k+1}\Vert }^2\le \delta D_h\left(y^k,y^{k+1}\right)\le \xi \Vert y^k-y^{k-1}\Vert {\text{Δ}}_{k,k+1}.$

Then

$\Vert y^k-y^{k+1}\Vert \le \sqrt{\frac{2\xi }{\delta }{\text{Δ}}_{k,k+1}}{\Vert y^k-y^{k-1}\Vert }^{1/2}.$

Using the fact that $2\sqrt{\alpha \beta }\le \alpha +\beta$, we obtain

$2\Vert y^k-y^{k+1}\Vert \le \Vert y^k-y^{k-1}\Vert +\frac{2\xi }{\delta }{\text{Δ}}_{k,k+1}.$ (3.39)

Summing (3.39) over for $k=\tilde{k}+1,\cdots ,m$ yields

$2\underset{k=\tilde{k}+1}{\overset{m}{{\displaystyle \sum }}}\Vert y^{k+1}-y^k\Vert \le \underset{k=\tilde{k}+1}{\overset{m}{{\displaystyle \sum }}}\Vert y^k-y^{k-1}\Vert +\frac{2\xi }{\delta }{\text{Δ}}_{\tilde{k}+1,m+1}.$

Notice that $\phi \left({ℒ}_{\beta }^h\left(w^{m+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)>0$ from Definition 2.9. Rearranging terms and taking $m\to +\infty$ yield

$\underset{k=\tilde{k}+1}{\overset{+\infty }{{\displaystyle \sum }}}\Vert y^{k+1}-y^k\Vert \le \Vert y^{\tilde{k}+1}-y^{\tilde{k}}\Vert +\frac{2\xi }{\delta }\phi \left({ℒ}_{\beta }^h\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right),$ (3.40)

Therefore,

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}\Vert y^{k+1}-y^k\Vert <+\infty .$ (3.41)

Combining (3.22) and (3.41), we obtain

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}\Vert {\lambda }^{k+1}-{\lambda }^k\Vert <+\infty .$ (3.42)

Using (3.30), we obtain

$\begin{array}{c}\Vert x^{k+1}-x^k\Vert \le \sqrt{\frac{3}{\alpha {\beta }^{2}M}}({\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2+{\Vert {\lambda }^k-{\lambda }^{k-1}\Vert }^2+2{\beta }^2{L}_h^2{\Vert y^{k+1}-y^k\Vert }^2\\ +{2{\left(\alpha -1\right)}^2{\beta }^2{L}_h^2{\Vert y^k-y^{k+1}\Vert }^2)}^{1/2}\\ \le \sqrt{\frac{3}{\alpha {\beta }^{2}M}}(\Vert {\lambda }^{k+1}-{\lambda }^k\Vert +\Vert {\lambda }^k-{\lambda }^{k-1}\Vert +2{\beta }^2{L}_h^2\Vert y^{k+1}-y^k\Vert \\ +2{\left(\alpha -1\right)}^2{\beta }^2{L}_h^2\Vert y^k-y^{k+1}\Vert )\end{array}$

Combining this inequality with (3.41) and (3.42), we have

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}\Vert x^{k+1}-x^k\Vert <+\infty .$ (3.43)

Additionally, we note that

$\begin{array}{c}\Vert w^{k+1}-w^k\Vert ={\left({\Vert x^{k+1}-x^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2+{\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2\right)}^{1/2}\\ \le \Vert x^{k+1}-x^k\Vert +\Vert y^{k+1}-y^k\Vert +\Vert {\lambda }^{k+1}-{\lambda }^k\Vert .\end{array}$

Using (3.41) - (3.43), we can conclude that

$\underset{k=0}{\overset{+\infty }{{\displaystyle \sum }}}\Vert w^{k+1}-w^k\Vert <+\infty ,$

implying that $\left\{w^k\right\}$ is a Cauchy sequence and thus convergent. By Lemma 3.4, we complete the proof. □

Theorem 3.2 (Convergence rate) Let $\left\{w^k=\left(x^k,y^k,{\lambda }^k\right)\right\}$ be the sequence generated by the Generalized Bregman ADMM (1.6) and converge to $\left\{w^{*}=\left(x^{*},y^{*},{\lambda }^{*}\right)\right\}$. Assuming that ${ℒ}_{\beta }^h(\cdot )$ has the KL property at $\left(x^{*},y^{*},{\lambda }^{*}\right)$ with $\phi \left(s\right)=c{s}^{1-\theta },$ $\theta \in \left[0,1\right)$, $c>0$. Then, the following results hold:

i) If $\theta =0$, then the sequence $\left\{w^k=\left(x^k,y^k,{\lambda }^k\right)\right\}$ converges in a finite number of steps.

ii) If $\theta \in \left(0,\frac{1}{2}\right]$, then there exists $c_1>0$ and $\tau \in \left[0,1\right)$ such that

$\Vert \left(x^k,y^k,{\lambda }^k\right)-\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\Vert \le c_1{\tau }^k.$

iii) If $\theta \in \left(\frac{1}{2},1\right)$, then there exists $c_2>0$ such that

$\parallel \left(x^k,y^k,{\lambda }^k\right)-\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)\parallel \le c_2{k}^{\left(\theta -1\right)/\left(2\theta -1\right)}.$

Proof: When $\theta =0$, we have $\phi \left(s\right)=cs$ and ${\phi }^{\prime }\left(s\right)=c$. Suppose, by contradiction, that $\left\{w^k=\left(x^k,y^k,{\lambda }^k\right)\right\}$ does not converge in a finite number of steps. Then, the KL property at $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$ yields, for any sufficiently large $k$, $c\cdot d\left(0,\partial {ℒ}_{\beta }^h\left(w^k\right)\right)\ge 1$, which contradicts Lemma 3.3.

Next, let $\theta >0$ and set ${\text{Δ}}_k=\underset{i=k}{\overset{+\infty }{{\displaystyle \sum }}}\Vert y^{i+1}-y^i\Vert$ for $k\ge 0$. By the triangle inequality, we have ${\text{Δ}}_k\ge \Vert y^k-y^{\text{*}}\Vert$, which allows us to estimate ${\text{Δ}}_k$. With these notations, it follows from (3.40) that

${\text{Δ}}_{\tilde{k}+1}\le {\text{Δ}}_{\tilde{k}}-{\text{Δ}}_{\tilde{k}+1}+\frac{2\xi }{\delta }\phi \left({ℒ}_{\beta }^h\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right).$

By invoking the KL property of ${ℒ}_{\beta }^h(\cdot )$ at $\left(x^{\text{*}},y^{\text{*}},{\lambda }^{\text{*}}\right)$, we obtain

${\phi }^{\prime }\left({ℒ}_{\beta }^h\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)d\left(0,\partial {ℒ}_{\beta }^h\left(w^{\tilde{k}+1}\right)\right)\ge 1,$

which is equivalent to

${\left({ℒ}_{\beta }^h\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)}^{\theta }\le c\cdot \left(1-\theta \right)d\left(0,\partial {ℒ}_{\beta }^h\left(w^{\tilde{k}+1}\right)\right).$ (3.44)

Using Lemma 3.3, we get

$d\left(0,\partial {ℒ}_{\beta }^h\left(w^{\tilde{k}+1}\right)\right)\le \xi \Vert y^{\tilde{k}+1}-y^{\tilde{k}}\Vert =\xi \left({\text{Δ}}_{\tilde{k}}-{\text{Δ}}_{\tilde{k}+1}\right).$ (3.45)

Combining (3.44) and (3.45), we obtain that there exists $\gamma >0$ such that

$\phi \left({ℒ}_{\beta }^h\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)=c\cdot {\left({ℒ}_{\beta }^h\left(w^{\tilde{k}+1}\right)-{ℒ}_{\beta }^h\left(w^{\text{*}}\right)\right)}^{1-\theta }\le \gamma {\left({\text{Δ}}_{\tilde{k}}-{\text{Δ}}_{\tilde{k}+1}\right)}^{\left(1-\theta \right)/\theta },$

and then

${\text{Δ}}_{\tilde{k}+1}\le {\text{Δ}}_{\tilde{k}}-{\text{Δ}}_{\tilde{k}+1}+\frac{2\xi }{\delta }\gamma {\left({\text{Δ}}_{\tilde{k}}-{\text{Δ}}_{\tilde{k}+1}\right)}^{\left(1-\theta \right)/\theta }.$

Sequences satisfying such inequalities have been studied in Attouch and Bolte [37] . It follows that

• If $\theta \in \left(0,\frac{1}{2}\right]$, then there exists $c_1>0$ and $\tau \in \left[0,1\right)$, such that

$\Vert y^k-y^{\text{*}}\Vert \le c_1{\tau }^k.$ (3.46)

• If $\theta \in \left(\frac{1}{2},1\right)$, then there exists $c_2>0$, such that

$\Vert y^k-y^{\text{*}}\Vert \le c_2{k}^{\frac{\theta -1}{2\theta -1}}.$ (3.47)

Recalling that

$\Vert {\lambda }^{k+1}-{\lambda }^k\Vert \le L{L}_h\Vert y^{k+1}-y^k\Vert ,$

we obtain

$\Vert {\lambda }^k-{\lambda }^{\text{*}}\Vert \le L{L}_h\Vert y^k-y^{\text{*}}\Vert .$ (3.48)

Furthermore, from the relations

${\lambda }^k={\lambda }^{k-1}-\beta \left(\nabla h\left(\alpha \left(A{x}^k-b\right)\right)-\nabla h\left(-y^k-\left(\alpha -1\right)y^{k-1}\right)\right),$

and

$\nabla h\left(\alpha \left(A{x}^{\text{*}}-b\right)\right)-\nabla h\left(-\alpha y^{\text{*}}\right)=0,$

it follows that

$\begin{array}{l}\beta \left(\nabla h\left(\alpha \left(A{x}^k-b\right)\right)-\nabla h\left(\alpha \left(A{x}^{\text{*}}-b\right)\right)\right)\\ =\left({\lambda }^{k-1}-{\lambda }^{\text{*}}\right)+\left({\lambda }^{\text{*}}-{\lambda }^k\right)+\beta \left(\nabla h\left(-y^k-\left(\alpha -1\right)y^{k-1}\right)-\nabla h\left(-\alpha y^{\text{*}}\right)\right)\end{array}$

We multiply both sides of the above equation by $\frac{1}{\beta }$ at the same time

$\begin{array}{l}\nabla h\left(\alpha \left(A{x}^k-b\right)\right)-\nabla h\left(\alpha \left(A{x}^{\text{*}}-b\right)\right)\\ =\frac{1}{\beta }\left({\lambda }^{k-1}-{\lambda }^{\text{*}}\right)+\frac{1}{\beta }\left({\lambda }^{\text{*}}-{\lambda }^k\right)+\left(\nabla h\left(-y^k-\left(\alpha -1\right)y^{k-1}\right)-\nabla h\left(-\alpha y^{\text{*}}\right)\right)\end{array}$

Now combine the above equation with the 1-strong convexity of $h(\cdot )$, and then we can get the following

$\begin{array}{l}\alpha \Vert x_k-x^{\text{*}}\Vert \le \Vert \nabla h\left(\alpha \left(A{x}_k-b\right)\right)-\nabla h\left(\alpha \left(A{x}^{\text{*}}-b\right)\right)\Vert \\ \le \frac{1}{\beta }\Vert \left({\lambda }^{k-1}-{\lambda }^{\text{*}}\right)+\frac{1}{\beta }\left({\lambda }^{\text{*}}-{\lambda }^k\right)+\left(\nabla h\left(-y^k-\left(\alpha -1\right)y^{k-1}\right)-\nabla h\left(-\alpha y^{\text{*}}\right)\right)\Vert \\ \le \frac{1}{\beta }\Vert {\lambda }^{k-1}-{\lambda }^{\text{*}}\Vert +\frac{1}{\beta }\Vert {\lambda }^{\text{*}}-{\lambda }^k\Vert +L_h\Vert y^{k-1}-y^k\Vert +\alpha L_h\Vert y^{k-1}-y^{\text{*}}\Vert \\ \le \frac{L{L}_h}{\beta }\Vert y^{k-1}-y^{\text{*}}\Vert +\frac{L{L}_h}{\beta }\Vert y^{\text{*}}-y^k\Vert +L_h\Vert y^{k-1}-y^k\Vert +\alpha L_h\Vert y^{k-1}-y^{\text{*}}\Vert \\ \le \frac{L{L}_h}{\beta }\Vert y^{k-1}-y^{\text{*}}\Vert +\frac{L{L}_h}{\beta }\Vert y^{\text{*}}-y^k\Vert +L_h\Vert y^{k-1}-y^{\text{*}}\Vert +L_h\Vert y^{\text{*}}-y^k\Vert +\alpha L_h\Vert y^{k-1}-y^{\text{*}}\Vert \\ =\left(\frac{L{L}_h}{\beta }+L_h+\alpha L_h\right)\Vert y_{k-1}-y^{\text{*}}\Vert +\left(\frac{L{L}_h}{\beta }+L_h\right)\Vert y^k-y^{\text{*}}\Vert .\end{array}$ (3.49)

Combining (3.48) and (3.49), we immediately obtain the desired inequalities from (3.46) and (3.47).

4. Conclusion

In this paper, we primarily analyze the generalized Bregman alternating direction method of multipliers (ADMM) for solving nonconvex separable problems subject to linear constraints. In contrast to the classical alternating direction method of multipliers, we modify the iterative format of the second subproblem. This modification relaxes the condition of global Lipschitz continuity for the gradient of differentiable functions. Additionally, we introduce a relaxation parameter $\alpha$, inspired by the acceleration technique of the proximal point algorithm (PPA), to enhance the algorithm’s performance. Under the assumption that the augmented Lagrangian function satisfies the Kurdyka-Lojasiewicz inequality, we prove that when the penalty parameters in the augmented Lagrangian function are sufficiently large, the iterative sequence generated by the algorithm converges to a critical point of the augmented Lagrangian function. Lastly, we set the corresponding parameters to further analyze the convergence rate of the algorithm.

NOTES

*Corresponding author.

References

1. Figueiredo, M.A.T. and Bioucas-Dias, J.M. (2010) Restoration of Poissonian Images Using Alternating Direction Optimization. IEEE Transactions on Image Processing, 19, 3133-3145. >https://doi.org/10.1109/tip.2010.2053941

2. Goldstein, T., Bresson, X. and Osher, S. (2009) Geometric Applications of the Split Bregman Method: Segmentation and Surface Reconstruction. Journal of Scientific Computing, 45, 272-293. >https://doi.org/10.1007/s10915-009-9331-z

3. Candes, E.J. and Tao, T. (2006) Near-Optimal Signal Recovery from Random Projections: Universal Encoding Strategies? IEEE Transactions on Information Theory, 52, 5406-5425. >https://doi.org/10.1109/tit.2006.885507

4. Beck, A. and Teboulle, M. (2009) Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems. IEEE Transactions on Image Processing, 18, 2419-2434. >https://doi.org/10.1109/tip.2009.2028250

5. Boyd, S. (2010) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends in Machine Learning, 3, 1-122. >https://doi.org/10.1561/2200000016

6. Dai, W. and Milenkovic, O. (2009) Subspace Pursuit for Compressive Sensing Signal Reconstruction. IEEE Transactions on Information Theory, 55, 2230-2249. >https://doi.org/10.1109/tit.2009.2016006

7. Yin, W., Osher, S., Goldfarb, D. and Darbon, J. (2008) Bregman Iterative Algorithms for $\ell_1$-Minimization with Applications to Compressed Sensing. SIAM Journal on Imaging Sciences, 1, 143-168. >https://doi.org/10.1137/070703983

8. Gabay, D. and Mercier, B. (1976) A Dual Algorithm for the Solution of Nonlinear Variational Problems via Finite Element Approximation. Computers&Mathematics with Applications, 2, 17-40. >https://doi.org/10.1016/0898-1221(76)90003-1

9. Glowinski, R. and Marroco, A. (1975) Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Revue française d’automatique, informatique, recherche opérationnelle. Analyse numérique, 9, 41-76. >https://doi.org/10.1051/m2an/197509r200411

10. He, B. and Yuan, X. (2014) On Non-Ergodic Convergence Rate of Douglas–Rachford Alternating Direction Method of Multipliers. Numerische Mathematik, 130, 567-577. >https://doi.org/10.1007/s00211-014-0673-6

11. He, B. and Yuan, X. (2012) On the O(1/n) Convergence Rate of the Douglas–Rachford Alternating Direction Method. SIAM Journal on Numerical Analysis, 50, 700-709. >https://doi.org/10.1137/110836936

12. Hong, M. and Luo, Z. (2016) On the Linear Convergence of the Alternating Direction Method of Multipliers. Mathematical Programming, 162, 165-199. >https://doi.org/10.1007/s10107-016-1034-2

13. Boley, D. (2013) Local Linear Convergence of the Alternating Direction Method of Multipliers on Quadratic or Linear Programs. SIAM Journal on Optimization, 23, 2183-2207. >https://doi.org/10.1137/120878951

14. Han, D. and Yuan, X. (2013) Local Linear Convergence of the Alternating Direction Method of Multipliers for Quadratic Programs. SIAM Journal on Numerical Analysis, 51, 3446-3457. >https://doi.org/10.1137/120886753

15. Yang, W.H. and Han, D. (2016) Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems. SIAM Journal on Numerical Analysis, 54, 625-640. >https://doi.org/10.1137/140974237

16. Li, G. and Pong, T.K. (2015) Global Convergence of Splitting Methods for Nonconvex Composite Optimization. SIAM Journal on Optimization, 25, 2434-2460. >https://doi.org/10.1137/140998135

17. Hong, M., Luo, Z. and Razaviyayn, M. (2016) Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems. SIAM Journal on Optimization, 26, 337-364. >https://doi.org/10.1137/140990309

18. Guo, K., Han, D.R. and Wu, T.T. (2016) Convergence of Alternating Direction Method for Minimizing Sum of Two Nonconvex Functions with Linear Constraints. International Journal of Computer Mathematics, 94, 1653-1669. >https://doi.org/10.1080/00207160.2016.1227432

19. Gabay, D. (1983) Applications of the Method of Multipliers to Variational Inequalities. In: Studies in Mathematics and Its Applications, Elsevier, 299-331. >https://doi.org/10.1016/s0168-2024(08)70034-1

20. Lions, P.L. and Mercier, B. (1979) Splitting Algorithms for the Sum of Two Nonlinear Operators. SIAM Journal on Numerical Analysis, 16, 964-979. >https://doi.org/10.1137/0716071

21. Eckstein, J. and Bertsekas, D.P. (1992) On the Douglas-Rachford Splitting Method and the Proximal Point Algorithm for Maximal Monotone Operators. Mathematical Programming, 55, 293-318. >https://doi.org/10.1007/bf01581204

22. Wang, X. and Guo, K. (2018) Convergence of the Generalized Alternating Direction Method of Multipliers for a Class of Nonconvex Optimization Problems. Applied Mathematics and Mechanics, 39, 1410-1425.

23. Bertsekas, D.P. (2014) Constrained Optimization and Lagrange Multiplier Methods. Academic Press.

24. Cai, X., Chen, Y. and Han, D. (2013) Nonnegative Tensor Factorizations Using an Alternating Direction Method. Frontiers of Mathematics in China, 8, 3-18. >https://doi.org/10.1007/s11464-012-0264-8

25. Eckstein, J. (1994) Parallel Alternating Direction Multiplier Decomposition of Convex Programs. Journal of Optimization Theory and Applications, 80, 39-62. >https://doi.org/10.1007/bf02196592

26. Bauschke, H.H., Bolte, J. and Teboulle, M. (2017) A Descent Lemma beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications. Mathematics of Operations Research, 42, 330-348. >https://doi.org/10.1287/moor.2016.0817

27. Bolte, J., Sabach, S., Teboulle, M. and Vaisbourd, Y. (2018) First Order Methods Beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems. SIAM Journal on Optimization, 28, 2131-2151. >https://doi.org/10.1137/17m1138558

28. Dragomir, R., d’Aspremont, A. and Bolte, J. (2021) Quartic First-Order Methods for Low-Rank Minimization. Journal of Optimization Theory and Applications, 189, 341-363. >https://doi.org/10.1007/s10957-021-01820-3

29. Nesterov, Y. (2019) Implementable Tensor Methods in Unconstrained Convex Optimization. Mathematical Programming, 186, 157-183. >https://doi.org/10.1007/s10107-019-01449-1

30. Tan, L. and Guo, K. (2025) Bregman ADMM: A New Algorithm for Nonconvex Optimization with Linear Constraints. Journal of Nonlinear and Variational Analysis, 9, 179-196.

31. Beck, A. (2017). First-Order Methods in Optimization. Society for Industrial and Applied Mathematics. >https://doi.org/10.1137/1.9781611974997

32. Bregman, L.M. (1967) The Relaxation Method of Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming. USSR Computational Mathematics and Mathematical Physics, 7, 200-217. >https://doi.org/10.1016/0041-5553(67)90040-7

33. Bauschke, H.H., Borwein, J.M. and Combettes, P.L. (2003) Bregman Monotone Optimization Algorithms. SIAM Journal on Control and Optimization, 42, 596-636. >https://doi.org/10.1137/s0363012902407120

34. Attouch, H., Bolte, J., Redont, P. and Soubeyran, A. (2010) Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality. Mathematics of Operations Research, 35, 438-457. >https://doi.org/10.1287/moor.1100.0449

35. Attouch, H., Bolte, J. and Svaiter, B.F. (2011) Convergence of Descent Methods for Semi-Algebraic and Tame Problems: Proximal Algorithms, Forward-Backward Splitting, and Regularized Gauss-Seidel Methods. Mathematical Programming, 137, 91-129. >https://doi.org/10.1007/s10107-011-0484-9

36. Bolte, J., Sabach, S. and Teboulle, M. (2013) Proximal Alternating Linearized Minimization for Nonconvex and Non-Smooth Problems. Mathematical Programming, 146, 459-494. >https://doi.org/10.1007/s10107-013-0701-9

37. Attouch, H. and Bolte, J. (2007) On the Convergence of the Proximal Algorithm for Non-Smooth Functions Involving Analytic Features. Mathematical Programming, 116, 5-16. >https://doi.org/10.1007/s10107-007-0133-5