Convergence of Bregman Alternating Direction Method of Multipliers for Nonseparable Nonconvex Objective with Linear Constraints

In this paper, our focus lies on addressing a two-block linearly constrained nonseparable nonconvex optimization problem with coupling terms. The most classical algorithm, the alternating direction method of multipliers (ADMM), is employed to solve such problems typically, which still requires the assumption of the gradient Lipschitz continuity condition on the objective function to ensure overall convergence from the current knowledge. However, many practical applications do not adhere to the conditions of smoothness. In this study, we justify the convergence of variant Bregman ADMM for the problem with coupling terms to circumvent the issue of the global Lipschitz continuity of the gradient. We demonstrate that the iterative sequence generated by our approach converges to a critical point of the issue when the corresponding function fulfills the Kurdyka-Lojasiewicz inequality and certain assumptions apply. In addition, we illustrate the convergence rate of the algorithm.

• Nonseparable Nonconvex Optimization
• Bregman ADMM
• Kurdyka-Lojasiewicz Inequality

1. Introduction

We consider the following two-block linearly constrained nonseparable nonconvex optimization problem:

$\begin{array}{ll}\min \mathrm{<mi>\; </mi>}\hfill & f\left(x\right)+g\left(y\right)+H\left(x\mathrm{,}y\right)\hfill \\ \text{s}\text{.t}\text{.}\hfill & Ax+y=b,\hfill \end{array}$ (1)

where $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is a proper lower semicontinuous function, $g\mathrm{<mo>\; :\; </mo>}{ℝ}^m\to ℝ$ is a continuous differentiable function, $H\mathrm{<mo>\; :\; </mo>}{ℝ}^n\times {ℝ}^m\to ℝ$ is a smooth function, $A\in {ℝ}^{m\times n}$ is a matrix and $b\in {ℝ}^m$ is a vector. Problem (1) finds numerous applications in scenarios such as the control of a smart grid system [1] [2] [3] , the appliance load model [4] , cognitive radio network and other related domains [5] [6] .

To solve this problem, the commonly employed method is the Alternating Direction Method of Multipliers (ADMM) [7] which stands out as a popular and well-established approach.

In 2017, Guo et al. studied the convergence of the ADMM to solve the problem (1) [7] and its iteration is as follows:

$(\begin{array}{l}{x}_{k+1}=\arg \underset{x}{\min }\left\{{ℒ}_{\beta }\left(x,y_k,{\lambda }_k\right)\right\},\\ y_{k+1}=\arg \underset{y}{\min }\left\{{ℒ}_{\beta }\left(x_{k+1},y,{\lambda }_k\right)\right\},\\ {\lambda }_{k+1}={\lambda }_k+\beta \left(A{x}_{k+1}+y_{k+1}-b\right).\end{array}$ (2)

Here, ${ℒ}_{\beta }(\cdot )$ denotes the augmented Lagrangian function defined as

$\begin{array}{c}{ℒ}_{\beta }\left(x\mathrm{,}y\mathrm{,}\lambda \right)=f\left(x\right)+g\left(y\right)+H\left(x\mathrm{,}y\right)+{\lambda }^{\text{T}}\left(Ax+y-b\right)\\ +\frac{\beta }{2}{\Vert Ax+y-b\Vert }_2^2\mathrm{,}\end{array}$

where $\lambda$ is the Lagrangian multiplier associated with the linear constraints and $\beta >0$ is the penalty parameter.

Note that, there is a critical condition that the function $g(\cdot )$ is L-smooth in their convergence analysis. Indeed, for most algorithms, the absence of convexity makes the smoothness condition a necessary requirement for convergence analysis. Unfortunately, the same holds true for ADMM [8] [9] [10] [11] . However, there exists a multitude of applications that are nonsmooth. The prevalence of non-smooth problems necessitates contemplating how to relax smoothness in nonconvex situations.

To alleviate the L-smoothness condition, Tan and Guo introduced in 2023 the convergence of the following Bregman ADMM for the special case of problem (1) [12] where the coupling term $H(\cdot )=0$:

$\begin{array}{ll}\min \mathrm{<mi>\; </mi>}\hfill & f\left(x\right)+g\left(y\right)\hfill \\ \text{s}\text{.t}\text{.}\hfill & Ax+y=b,\hfill \end{array}$ (3)

where $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is a proper lower semicontinuous function, $g\mathrm{<mo>\; :\; </mo>}{ℝ}^m\to ℝ$ is a continuous differentiable function, $A\in {ℝ}^{m\times n}$ is a matrix and $b\in {ℝ}^m$ is a vector. It also encompasses a range of significant applications in diverse areas, such as imaging processing [13] [14] [15] , statistical learning [16] [17] [18] , engineering [19] [20] [21] and so on. Problem (1) reduces to (3) when the coupled function H is absent.

The iterative format of Bregman ADMM is as follows:

$(\begin{array}{l}{x}_{k+1}=\arg \underset{x}{\min }\left\{{ℒ}_{\beta }^h\left(x,y_k,{\lambda }_k\right)\right\},\\ y_{k+1}=\arg \underset{y}{\min }\left\{{ℒ}_{\beta }^h\left(x_{k+1},y,{\lambda }_k\right)\right\},\\ {\lambda }_{k+1}={\lambda }_k+\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right).\end{array}$ (4)

Here, the augmented Lagrangian function ${ℒ}_{\beta }^h(\cdot )$ which is defined as

${ℒ}_{\beta }^h\left(x\mathrm{,}y\mathrm{,}\lambda \right)=f\left(x\right)+g\left(y\right)+{\lambda }^{\text{T}}\left(Ax+y-b\right)+\beta D_h\left(-y\mathrm{,}Ax-b\right)\mathrm{.}$

where $\lambda$ is the Lagrangian multiplier associated with the linear constraints and $\beta >0$ is the penalty parameter. The above Bregman ADMM reduces to the

classic ADMM when $h=\frac{1}{2}{\Vert \cdot \Vert }_2^2$. However, their proof only established the con-

vergence of the separable problem (3), where there is no coupling term. Therefore, the motivation of this paper is to extend the algorithm to the nonseparable problem (1) with a coupling term.

The iterative format of Bregman ADMM for the problem (1) is as follows:

$(\begin{array}{l}{x}_{k+1}=\arg \underset{x}{\min }\left\{{ℒ}_{\beta }^h\left(x,y_k,{\lambda }_k\right)\right\},\\ y_{k+1}=\arg \underset{y}{\min }\left\{{ℒ}_{\beta }^h\left(x_{k+1},y,{\lambda }_k\right)\right\},\\ {\lambda }_{k+1}={\lambda }_k+\beta \left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right).\end{array}$ (5)

Here, the augmented Lagrangian function ${ℒ}_{\beta }^h(\cdot )$ which is defined as

${ℒ}_{\beta }^h\left(x\mathrm{,}y\mathrm{,}\lambda \right)=f\left(x\right)+g\left(y\right)+H\left(x\mathrm{,}y\right)+{\lambda }^{\text{T}}\left(Ax+y-b\right)+\beta D_h\left(-y\mathrm{,}Ax-b\right)\mathrm{.}$

where $\lambda$ is the Lagrangian multiplier associated with the linear constraints and $\beta >0$ is the penalty parameter.

Given our aim to relax the strict smoothness condition $g(\cdot )$ to be relatively smooth, our focus is solely on modifying the Euclidean distance in the iterative formulation of the y variable to the Bregman distance, and the iterative format of x is the same as the classical ADMM. The iterative format called new Bregman ADMM for the problem (1) is as follows:

$(\begin{array}{l}{x}_{k+1}=\arg \underset{x}{\min }\left\{f\left(x\right)+g\left(y_k\right)+H\left(x,y_k\right)+{\lambda }_k^{\text{T}}\left(Ax+y_k-b\right)+\frac{{\beta }_1}{2}{\Vert Ax+y_k-b\Vert }^2\right\},\\ y_{k+1}=\arg \underset{y}{\min }\left\{{ℒ}_{{\beta }_2}^h\left(x_{k+1},y,{\lambda }_k\right)\right\},\\ {\lambda }_{k+1}={\lambda }_k+{\beta }_2\left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right).\end{array}$(6)

Here denotes ${ℒ}_{{\beta }_2}^h(\cdot )$ the augmented Lagrangian function which is defined as

${ℒ}_{{\beta }_2}^h\left(x\mathrm{,}y\mathrm{,}\lambda \right)=f\left(x\right)+g\left(y\right)+H\left(x\mathrm{,}y\right)+{\lambda }^{\text{T}}\left(Ax+y-b\right)+{\beta }_2{D}_h\left(-y\mathrm{,}Ax-b\right)\mathrm{.}$

where $\lambda$ is the Lagrangian multiplier associated with the linear constraints and ${\beta }_1,{\beta }_2>0$ is the penalty parameter.

Building upon the aforementioned motivation, our attention is directed towards the Bregman ADMM applied to the nonseparable problem (1). We delve into the following aspects: For two-block nonseparable nonconvex optimization problems with linear constraints, we develop an appropriate Lyapunov function and leverage the KL inequality to examine the global convergence of the Bregman ADMM. The paper is organized as follows. We first summarize some necessary preliminaries for further analysis in Section 2. In Section 3, we analyze the convergence of variant Bregman ADMM and its convergence rate. Finally, some conclusions are made in Section 4.

2. Preliminaries

In this section, we summarized some notations and preliminaries to be used for further analysis.

Definition 2.1. [22] For an extended real-valued function $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$, the effective domain or just the domain is the set

$dom\left(f\right)=\left\{x\in {ℝ}^n\mathrm{<mo>\; :\; </mo>}f\left(x\right)<\infty \right\}\mathrm{.}$

Definition 2.2. [22] A function $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is called proper if there at least one $x\in {ℝ}^n$ such that $f\left(x\right)<\infty$.

Definition 2.3. [22] A function $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is called lower semicontinuous at $x\in {ℝ}^n$ if

$f\left(x\right)\le \underset{k\to \infty }{lim}\inf f\left(x_k\right)$

for any sequence $\left\{x_k\right\}\subseteq {ℝ}^n$ for which $x_k\to x$ as $k\to \infty$. Moreover, f is called lower semicontinuous if it is lower semicontinuous at each point in ${ℝ}^n$.

Definition 2.4. ( [23] kernel generating distance) Let C be a nonempty, convex, and open subset of ${ℝ}^m$. Associated with C, a function $h\mathrm{<mo>\; :\; </mo>}{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$ is called a kernel generating distance if it satisfies the following:

1) $h$ is proper, lower semicontinuous, and convex, with $dom\left(h\right)\subset \bar{C}$ and $dom\left(\partial h\right)=C$.

2) $h$ is $C^1$ on $int\left(dom\left(h\right)\right)\equiv C$.

We denote the class of kernel generating distance by $\mathcal{G}\left(C\right)$.

Given $h\in \mathcal{G}\left(C\right)$, define Bregman distance $D_h:\text{dom}\left(h\right)\times \mathrm{int}\left(\text{dom}\left(h\right)\right)\to {ℝ}^{+}$ by

$D_h\left(x\mathrm{,}y\right)\mathrm{<mo>\; :\; </mo>}=h\left(x\right)-h\left(y\right)-\langle \nabla h\left(y\right)\mathrm{,}x-y\rangle \mathrm{.}$ (7)

Since h is convex, $D_h\left(x\mathrm{,}y\right)\ge 0$, and $D_h\left(x,y\right)=0$ only when $x=y$.

Lemma 2.1. [24] Let $h\mathrm{<mo>\; :\; </mo>}{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$ be a kernel generating distance. The following properties follow directly from (7). Let $x\mathrm{,}y\in int\left(dom\left(h\right)\right)$ and $z\in dom\left(h\right)$, then

1) $D_h\left(x\mathrm{,}y\right)+D_h\left(y\mathrm{,}x\right)=\langle \nabla h\left(x\right)-\nabla h\left(y\right)\mathrm{,}x-y\rangle$.

2) Three points identity holds:

$D_h\left(z\mathrm{,}x\right)-D_h\left(z\mathrm{,}y\right)-D_h\left(y\mathrm{,}x\right)=\langle \nabla h\left(x\right)-\nabla h\left(y\right)\mathrm{,}y-z\rangle \mathrm{.}$ (8)

In the subsequent analysis, we will use a pair of functions $\left(g\mathrm{,}h\right)$ satisfying

1) $h\in \mathcal{G}\left(C\right)$.

2) $g\mathrm{<mo>\; :\; </mo>}{ℝ}^m\cup \left\{+\infty \right\}$ is a proper and lower semicontinuous function $\text{dom}\left(h\right)\subset \text{dom}\left(g\right)$, which is continuously differentiable on $C=\mathrm{int}\left(\text{dom}\left(h\right)\right)$.

Definition 2.5. ( [23] L-smooth adaptable) A pair $\left(g\mathrm{,}h\right)$ is called L-smooth adaptable on C if there exists $L>0$ such that $Lh-g$ and $Lh+g$ are convex on C.

Remark 2.1. Since $Lh-g$ and $Lh+g$ is convex, then its gradient are monotonous, we obtain

$\langle \nabla g\left(x\right)-\nabla g\left(y\right)\mathrm{,}x-y\rangle \le L\langle \nabla h\left(x\right)-\nabla h\left(y\right)\mathrm{,}x-y\rangle =L\left(D_h\left(x\mathrm{,}y\right)+D_h\left(y\mathrm{,}x\right)\right)\mathrm{,}$

and

$\langle \nabla g\left(y\right)-\nabla g\left(x\right)\mathrm{,}x-y\rangle \le L\langle \nabla h\left(x\right)-\nabla h\left(y\right)\mathrm{,}x-y\rangle =L\left(D_h\left(x\mathrm{,}y\right)+D_h\left(y\mathrm{,}x\right)\right)\mathrm{.}$

If we assume,

$\Vert \nabla g\left(x\right)-\nabla g\left(y\right)\Vert \le L\frac{D_h\left(x\mathrm{,}y\right)+D_h\left(y\mathrm{,}x\right)}{\Vert x-y\Vert }$, for all $x\ne y\in int\left(dom\left(h\right)\right)$, (9)

then by Cauchy-Schward inequality we immediately obtain the above two inequalities. Thus, (9) provides a Lipschitz-like gradient property of g with respect to $D_h$.

Remark 2.2. Definition 2.5 naturally complements and extends the definition of “A Lipschitz-like/Convexity Condition” in [22] , which allows to obtain the following two-sided descend lemma.

Lemma 2.2. ( [23] extended descent lemma) The pair of functions $\left(g\mathrm{,}h\right)$ is L-smooth adaptable on C if and only if

$\left|g\left(x\right)-g\left(y\right)-\langle \nabla g\left(y\right)\mathrm{,}x-y\rangle \right|\le L{D}_h\left(x\mathrm{,}y\right)\mathrm{<mi>\; </mi>\; <mi>\; </mi>}\forall x\mathrm{,}y\in int\left(dom\left(h\right)\right)\mathrm{.}$ (10)

In particular, when the set $C={ℝ}^m$ and $h(\cdot )=\left(1/2\right){\Vert \cdot \Vert }^2$, (10) reduces to the classical descent lemma for a function g with a L-smooth gradient on ${ℝ}^m$, i.e.,

$\left|g\left(x\right)-g\left(y\right)-\langle \nabla g\left(y\right)\mathrm{,}x-y\rangle \right|\le \frac{L}{2}\cdot {\Vert x-y\Vert }^2\mathrm{<mi>\; </mi>\; <mi>\; </mi>\; <mi>\; </mi>}\forall x\mathrm{,}y\in {ℝ}^m\mathrm{.}$

Definition 2.6. [23] Let $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper lower semicontinuous function.

1) The Fréchet subdifferential, or regular subdifferential of f at $x\in dom\left(f\right)$, written $\hat{\partial }f\left(x\right)$, is the set of vectors $x^{\mathrm{<mo>\; *\; </mo>}}\in {ℝ}^n$ that satisfy

$\underset{y\ne x}{lim}\underset{y\to x}{\inf }\frac{f\left(y\right)-f\left(x\right)-\langle x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}y-x\rangle }{\Vert y-x\Vert }\ge 0.$

When $x\notin dom\left(f\right)$we set $\hat{\partial }f\left(x\right)=\varnothing$.

2) The limiting-subdifferential, or simply the subdifferential, of $f$ at $x\in dom\left(f\right)$, written $\partial f\left(x\right)$, is defined as follows:

$\partial f\left(x\right)=\left\{x^{*}\in {ℝ}^n,\exists x_n\to x,f\left(x_n\right)\to f\left(x\right),x_n^{*}\in \hat{\partial }f\left(x_n\right),with{x}_n^{*}\to x^{*}\right\}.$

Remark 2.3. From the above definition, we note that

1) The above definition implies $\hat{\partial }f\left(x\right)\subseteq \partial f\left(x\right)$ for each $x\in {ℝ}^n$, where the first set is closed convex while the second one is only closed.

2) Let $\left(x_k\mathrm{,}{x}_k^{\mathrm{<mo>\; *\; </mo>}}\right)\in Graph\mathrm{<mi>\; </mi>}\partial f$ be a sequence that converges to $\left(x\mathrm{,}{x}^{\mathrm{<mo>\; *\; </mo>}}\right)$. By the definition of $\partial f$, if $f\left(x_k\right)$ converges to $f\left(x\right)$ as $k\to +\infty$, then $\left(x\mathrm{,}{x}^{\mathrm{<mo>\; *\; </mo>}}\right)\in Graph\mathrm{<mi>\; </mi>}\partial f$.

3) A necessary condition for $x\in {ℝ}^n$ to be a minimizer of $f$ is

$0\in \partial f\left(x\right)\mathrm{.}$ (11)

4) If $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is a proper lower semicontinuous and $g\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ$ is continuous differentiable, then $\partial \left(f+g\right)\left(x\right)=\partial f\left(x\right)+\nabla g\left(x\right)$ for any $x\in dom\left(f\right)$.

A point satisfying Equation (11) is called a critical point or a station point. The critical points set of f is denoted by crit $f$.

Now, we recall an important property of subdifferential calculus.

Lemma 2.3. [25] Suppose that $F\left(x\mathrm{,}y\right)=f\left(x\right)+g\left(x\right)$, where $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ and $g\mathrm{<mo>\; :\; </mo>}{ℝ}^m\to ℝ\cup \left\{+\infty \right\}$ are proper lower $r$ semicontinuous functions. Then for all $\left(x,y\right)\in dom\left(F\right)=dom\left(f\right)\times dom\left(g\right)$, we have

$\partial F\left(x,y\right)={\partial }_{x}F\left(x,y\right)\times {\partial }_{y}F\left(x,y\right).$

Definition 2.7. ( [25] Kurdyka-Lojasiewicz inequality) Let $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper lower semicontinuous function. For $-\infty <{\eta }_1<{\eta }_1\le +\infty$, set

$\left[{\eta }_1<f<{\eta }_1\right]=\left\{x\in {ℝ}^n:{\eta }_1<f\left(x\right)<{\eta }_2\right\}.$

We say that function f has the KL property at $x^{\mathrm{<mo>\; *\; </mo>}}\in dom\left(\partial f\right)$ if there exist $\eta \in \left(\mathrm{0,}+\infty \right]$, a neighbourhood U of $x^{\mathrm{<mo>\; *\; </mo>}}$, and a continuous concave function $\phi \mathrm{<mo>\; :\; </mo>}\left[\mathrm{0,}\eta \right)\to {ℝ}_{+}$, such that

1) $\phi \left(0\right)=0$;

2) $\phi$ is $C^1$ on $\left(\mathrm{0,}\eta \right)$ and continuous at 0;

3) ${\phi }^{\prime }\left(x\right)>0,\mathrm{<mi>\; </mi>}\forall x\in \left(0,\eta \right)$;

4) for all $x$ in $U\cap \left[f\left(x^{*}\right)<f<f\left(x^{*}\right)+\eta \right]$, the Kurdyka-Lojasiewicz inequality holds

${\phi }^{\prime }\left(f\left(x\right)-f\left(x^{\mathrm{<mo>\; *\; </mo>}}\right)\right)d\left(\mathrm{0,}\partial f\left(x\right)\right)\ge 1.$

Definition 2.8. ( [26] Kurdyka-Lojasiewicz function) Denote ${\varphi }_{\eta }$ be the set of functions that satisfy Defination 2.7 1) - 3). If f satisfies the KL property at each point of $dom\left(\partial f\right)$, then $f$ is called a KL function.

Lemma 2.4 ( [27] Uniformized KL property) Let Ω be a compact set and let $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a proper and lower semicontinuous function. Assume that f is constant on Ω and satisfies the KL property at each point of Ω. Then, there exist $\epsilon >0$, $\eta >0$, and $\phi \in {\varphi }_{\eta }$ such that for all $\bar{x}\in \Omega$ and for all x in the following intersection:

$\left\{x\in {ℝ}^n:d\left(x,\Omega \right)<\epsilon \right\}\cap \left[f\left(\bar{x}\right)<f<f\left(\bar{x}\right)+\eta \right].$

one has

${\phi }^{\prime }\left(f\left(x\right)-f\left(\bar{x}\right)\right)d\left(\mathrm{0,}\partial f\left(x\right)\right)\ge 1.$

Definition 2.9. We say that $\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)$ is a critical point of the Augmented Lagrangian Function with Bregman distance ${ℒ}_{\beta }(\cdot )$ if it satisfies

$\{\begin{array}{l}-A^{\text{T}}{\lambda }^{*}\in \partial f\left(x^{*}\right),\\ -{\lambda }^{*}=\nabla g\left(y^{*}\right),\\ A{x}^{*}+y^{*}-b=0.\end{array}$

Lemma 2.5. [28] Let $h\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ$ be a continuously differentiable function whose gradient $\nabla h$ is Lipschitz continuous with constant $L>0$, then for any $x\mathrm{,}y\in {ℝ}^n$, we have

$\left|h\left(y\right)-h\left(x\right)-\langle \nabla h\left(x\right)\mathrm{,}y-x\rangle \right|\le \frac{L}{2}{\Vert y-x\Vert }^2\mathrm{.}$

Definition 2.10. A proper lower semicontinuous function $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ is called semi-convex with constant $\omega \ge 0$ if the function

$x\mapsto f\left(x\right)+\frac{\omega }{2}{\Vert x\Vert }^2$

is convex. Specially, if $\omega =0$, then g is convex.

Remark 2.4. Definition (2.10) is equivalent to

$f\left(y\right)\ge f\left(x\right)+\langle p\mathrm{,}y-x\rangle -\frac{\omega }{2}{\Vert y-x\Vert }^2\mathrm{,}$

for all $x\mathrm{,}y\in {ℝ}^n$ and all $p\in \partial f\left(x\right)$.

3. Convergence Analysis

We commence our analysis by examining the optimality condition. By invoking the optimality condition for Equation (6), we obtain

$(\begin{array}{l}0\in \partial f\left(x_{k+1}\right)+{\nabla }_{x}H\left(x_{k+1},y_k\right)+A^{\text{T}}{\lambda }_k+{\beta }_1{A}^{\text{T}}\left(A{x}_{k+1}+y_k-b\right)\\ 0=\nabla g\left(y_{k+1}\right)+{\nabla }_{y}H\left(x_{k+1},y_{k+1}\right)+{\lambda }_k+{\beta }_2\left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)\\ {\lambda }_{k+1}={\lambda }_k+{\beta }_2\left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right).\end{array}$ (12)

By utilizing the previous equation and reorganizing terms, we derive

$(\begin{array}{l}-{\nabla }_{x}H\left(x_{k+1},y_k\right)-A^{\text{T}}{\lambda }_k-{\beta }_1{A}^{\text{T}}\left(A{x}_{k+1}+y_k-b\right)\in \partial f\left(x_{k+1}\right)\\ -{\nabla }_{y}H\left(x_{k+1},y_{k+1}\right)-{\lambda }_k-{\beta }_2\left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)=\nabla g\left(y_{k+1}\right)\\ {\lambda }_{k+1}={\lambda }_k+{\beta }_2\left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right).\end{array}$ (13)

In what follows, we assume:

Assumption 3.1. Let $f\mathrm{<mo>\; :\; </mo>}{ℝ}^n\to ℝ\cup \left\{+\infty \right\}$ be a semiconvex function with constant $\omega >0$, $\left(g\mathrm{,}h\right)$ be L-smooth adaptable on domh where h be $C^2$ on the interior of domh, ${\mu }_h$ strong-convex and $\nabla h$ is Lipschitz continuous with $L_h$ on any bounded subset of ${ℝ}^m$, and let $H\mathrm{<mo>\; :\; </mo>}{ℝ}^n\times {ℝ}^m\to ℝ$ be a smooth function. Assume the following holds:

1) ${\inf }_{\left(x,y\right)\in {ℝ}^n\times {ℝ}^m}H\left(x,y\right)>-\infty$, ${\inf }_{x\in {ℝ}^n}{f}_1\left(x\right)>-\infty$, ${\inf }_{y\in {ℝ}^m}{f}_2\left(y\right)>-\infty$;

2) For any fixed x, the partial gradient ${\nabla }_{y}H\left(x\mathrm{,}y\right)$ is globally Lipschitz with constant $L_2\left(x\right)$, that is

$\Vert {\nabla }_{y}H\left(x\mathrm{,}{y}_1\right)-{\nabla }_{y}H\left(x\mathrm{,}{y}_2\right)\Vert \le L_2\left(x\right)\Vert y_1-y_2\Vert \mathrm{,}\forall y_1\mathrm{,}{y}_2\in {ℝ}^m\mathrm{.}$

For any fixed y, the partial gradient ${\nabla }_{x}H\left(x\mathrm{,}y\right)$ is globally Lipschitz with constant $L_3\left(y\right)$, that is

$\Vert {\nabla }_{x}H\left(x_1\mathrm{,}y\right)-{\nabla }_{x}H\left(x_2\mathrm{,}y\right)\Vert \le L_3\left(y\right)\Vert x_1-x_2\Vert \mathrm{,}\forall x_1\mathrm{,}{x}_2\in {ℝ}^n\mathrm{.}$

3) There exists $L_2,L_3>0$ such that

$\sup \left\{L_2\left(x_k\right)\mathrm{<mo>\; :\; </mo>}k\in N\right\}\le L_2\mathrm{,}\sup \left\{L_3\left(y_k\right)\mathrm{<mo>\; :\; </mo>}k\in N\right\}\le L_3\mathrm{.}$

4) $\nabla H$ is Lipschitz continuous on bounded subsets of ${ℝ}^n\times {ℝ}^m$. In other words, for each bounded subset $B_1\times B_2\subseteq {ℝ}^n\times {ℝ}^m$, there exists $M>0$ such that for all $\left(x_i\mathrm{,}{y}_i\right)\in B_1\times B_2$, $i=1,2$:

$\begin{array}{l}\Vert \left({\nabla }_{x}H\left(x_1\mathrm{,}{y}_1\right)-{\nabla }_{x}H\left(x_2\mathrm{,}{y}_2\right)\mathrm{,}{\nabla }_{y}H\left(x_1\mathrm{,}{y}_1\right)-{\nabla }_{y}H\left(x_2\mathrm{,}{y}_2\right)\right)\Vert \\ \le M\Vert \left(x_1-x_2\mathrm{,}{y}_1-y_2\right)\Vert \mathrm{.}\end{array}$

5) $A^{\text{T}}A\succcurlyeq \mu I$ for some $\mu >0$.

6) ${\beta }_1=L_h{\beta }_2$

For the sake of convenience, in the following analysis, we frequently employ the notation ${\left\{w_k\mathrm{<mo>\; :\; </mo>}=\left(x_k\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)\right\}}_{k\in N}$ and ${\left\{v_k\mathrm{<mo>\; :\; </mo>}=\left(x_k\mathrm{,}{y}_k\right)\right\}}_{k\in N}$. We commence our analysis with the subsequent lemma.

Lemma 3.1. Let ${\left\{w_k\right\}}_{k\in N}$ be the sequence generated by the ADMM (1.3) which is assumed to be bounded, then we have

$ℒ\left(w_{k+1}\right)\le ℒ\left(w_k\right)-\delta {\Vert v_{k+1}-v_k\Vert }^2$ (14)

where

$\begin{array}{c}ℒ\left(x\mathrm{,}y\mathrm{,}\lambda \right)=f\left(x\right)+g\left(y\right)+H\left(x\mathrm{,}y\right)+{\lambda }^{\text{T}}\left(Ax+y-b\right)\\ +\frac{{\beta }_1}{2}{\Vert Ax+y-b\Vert }^2+{\beta }_2{D}_h\left(-y\mathrm{,}Ax-b\right)\end{array}$

Proof.

By considering the definition $ℒ(\cdot )$, we obtain

$\begin{array}{c}ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_{k+1}\right)=ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_k\right)+\langle {\lambda }_{k+1}-{\lambda }_k\mathrm{,}A{x}_{k+1}+y_{k+1}-b\rangle \mathrm{.}\\ \le ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_k\right)+\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \Vert A_{k+1}+y_{k+1}-b\Vert \mathrm{.}\end{array}$ (15)

By applying the fact that h is ${\mu }_h$-strong-convex, we get

$\langle \nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\mathrm{,}A{x}_{k+1}+y_{k+1}-b\rangle \ge {\mu }_h{\Vert A{x}_{k+1}+y_{k+1}-b\Vert }^2\mathrm{.}$ (16)

Using the Cauchy-Schwarz inequality, we have

$\begin{array}{l}\Vert \nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\Vert \Vert A{x}_{k+1}+y_{k+1}-b\Vert \\ \ge \langle \nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\mathrm{,}A{x}_{k+1}+y_{k+1}-b\rangle \mathrm{.}\end{array}$ (17)

Combining (16) and (17), we can conclude that

$\Vert \nabla h\left(-y_{k+1}\right)-\nabla h\left(A{x}_{k+1}-b\right)\Vert \ge {\mu }_h\Vert A{x}_{k+1}+y_{k+1}-b\Vert \mathrm{.}$ (18)

Based on the iterative scheme of the algorithm, we can infer the following

${\lambda }_{k+1}={\lambda }_k+{\beta }_2\left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)\mathrm{.}$

Therefore,

$\Vert \nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\Vert =\frac{1}{{\beta }_2}\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \mathrm{.}$ (19)

By combining Equations (18) and (19), we can infer that

$\Vert A{x}_{k+1}+y_{k+1}-b\Vert \le \frac{1}{{\beta }_2{\mu }_h}\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \mathrm{.}$ (20)

Substituting the previous inequality into Equation (15), which yields

$ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_{k+1}\right)\le ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_k\right)+\frac{1}{{\beta }_2{\mu }_h}{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2\mathrm{.}$ (21)

As defined by $ℒ(\cdot )$, we have

$\begin{array}{l}ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_k\right)\\ =ℒ\left(x_{k+1}\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)+g\left(y_{k+1}\right)-g\left(y_k\right)+H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)\\ -H\left(x_{k+1}\mathrm{,}{y}_k\right)+\langle {\lambda }_k\mathrm{,}{y}_{k+1}-y_k\rangle +{\beta }_2{D}_h\left(-y_{k+1}\mathrm{,}A{x}_{k+1}-b\right)\\ -{\beta }_2{D}_h\left(-y_k\mathrm{,}A{x}_{k+1}-b\right)+\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_{k+1}-b\Vert }^2\\ -\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_k-b\Vert }^2\mathrm{.}\end{array}$ (22)

Assumption A (3.1) demonstrates $\left(g\mathrm{,}h\right)$ is L-smooth adaptable, which implies

$g\left(y_{k+1}\right)-g\left(y_k\right)\le -\langle \nabla g\left(y_{k+1}\right)\mathrm{,}{y}_k-y_{k+1}\rangle +L{D}_h\left(y_k\mathrm{,}{y}_{k+1}\right)\mathrm{.}$ (23)

For any fixed $x$, the partial gradient ${\nabla }_{y}H\left(x\mathrm{,}y\right)$ is globally Lipschitz with constant $L_2\left(x\right)$, which can be inferred from assumptions 2). Further applying the Lemma (2.5) yields

$\begin{array}{l}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)-H\left(x_{k+1}\mathrm{,}{y}_k\right)\\ \le -\langle {\nabla }_{y}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)\mathrm{,}{y}_k-y_{k+1}\rangle +\frac{L_2\left(x_{k+1}\right)}{2}{\Vert y_{k+1}-y_k\Vert }^2\mathrm{.}\end{array}$ (24)

By three points identity (8), we obtain,

$\begin{array}{l}{\beta }_2{D}_h\left(-y_{k+1},A{x}_{k+1}-b\right)-{\beta }_2{D}_h\left(-y_k,A{x}_{k+1}-b\right)\\ ={\beta }_2{D}_h\left(-y_{k+1},A{x}_{k+1}-b\right)-{\beta }_2{D}_h\left(-y_k,A{x}_{k+1}-b\right)\\ +{\beta }_2{D}_h\left(-y_k,-y_{k+1}\right)-{\beta }_2{D}_h\left(-y_k,-y_{k+1}\right)\\ =-{\beta }_2(D_h\left(-y_k,A{x}_{k+1}-b\right)-D_h\left(-y_k,-y_{k+1}\right)\\ -D_h\left(-y_{k+1},A{x}_{k+1}-b\right)-{\beta }_2{D}_h\left(-y_k,-y_{k+1}\right)\end{array}$

$\begin{array}{l}=-{\beta }_2\langle \nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right),y_k-y_{k+1}\rangle \\ -{\beta }_2{D}_h\left(-y_k,-y_{k+1}\right)\\ =\langle {\lambda }_{k+1}-{\lambda }_k,y_{k+1}-y_k\rangle -{\beta }_2{D}_h\left(-y_k,-y_{k+1}\right).\end{array}$ (25)

The above two equations use that ${\lambda }_{k+1}={\lambda }_k+{\beta }_2\left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right).$

Inserting the above three Equations (24), (25), (23) into (22) yields

$\begin{array}{l}ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_k\right)-ℒ\left(x_{k+1}\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)\\ \le L{D}_h\left(y_k\mathrm{,}{y}_{k+1}\right)-{\beta }_2{D}_h\left(-y_k\mathrm{,}-y_{k+1}\right)+\frac{L_2}{2}{\Vert y_{k+1}-y_k\Vert }^2\\ +\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_{k+1}-b\Vert }^2-\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_k-b\Vert }^2+\langle {\lambda }_{k+1}-{\lambda }_k\mathrm{,}{y}_{k+1}-y_k\rangle \\ +\langle -\left(\nabla g\left(y_{k+1}\right)+{\nabla }_{y}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)\right)\mathrm{,}{y}_k-y_{k+1}\rangle +\langle {\lambda }_k\mathrm{,}{y}_{k+1}-y_k\rangle \end{array}$ (26)

From optimal condition, ${\lambda }_{k+1}=-\left(\nabla g\left(y_{k+1}\right)+{\nabla }_{y}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)\right)$ (13) we get

$\begin{array}{l}ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_k\right)-ℒ\left(x_{k+1}\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)\\ \le L{D}_h\left(y_k\mathrm{,}{y}_{k+1}\right)-{\beta }_2{D}_h\left(-y_k\mathrm{,}-y_{k+1}\right)+\frac{L_2}{2}{\Vert y_{k+1}-y_k\Vert }^2\\ +\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_{k+1}-b\Vert }^2-\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_k-b\Vert }^2\mathrm{.}\end{array}$ (27)

From the condition that h is ${\mu }_h$-strong-convex and $L_h$-smooth, we get

$\frac{{\mu }_h}{2}{\Vert y_{k+1}-y_k\Vert }^2\le D_h\left(-y_k\mathrm{,}-y_{k+1}\right)\le \frac{L_h}{2}{\Vert y_{k+1}-y_k\Vert }^2\mathrm{,}$

$\frac{{\mu }_h}{2}{\Vert y_{k+1}-y_k\Vert }^2\le D_h\left(y_k\mathrm{,}{y}_{k+1}\right)\le \frac{L_h}{2}{\Vert y_{k+1}-y_k\Vert }^2\mathrm{.}$ (28)

Substituting the aforementioned inequality into (27) reveals

$\begin{array}{l}ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_k\right)-ℒ\left(x_{k+1}\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)\\ \le \frac{L{L}_h+L_2-{\beta }_2{\mu }_h}{2}{\Vert y_{k+1}-y_k\Vert }^2+\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_{k+1}-b\Vert }^2-\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_k-b\Vert }^2\mathrm{.}\end{array}$ (29)

By defination of $ℒ(\cdot )$, we obtain

$\begin{array}{l}ℒ\left(x_{k+1}\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)-ℒ\left(x_k\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)\\ =f\left(x_{k+1}\right)-f\left(x_k\right)+H\left(x_{k+1}\mathrm{,}{y}_k\right)-H\left(x_k\mathrm{,}{y}_k\right)+\langle {\lambda }_k\mathrm{,}A{x}_{k+1}-A{x}_k\rangle \\ +{\beta }_2{D}_h\left(-y_k\mathrm{,}A{x}_{k+1}-b\right)-{\beta }_2{D}_h\left(-y_k\mathrm{,}A{x}_k-b\right)\\ +\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_k-b\Vert }^2-\frac{{\beta }_1}{2}{\Vert A{x}_k+y_k-b\Vert }^2\mathrm{.}\end{array}$ (30)

From the Assumption 3.1 that the partial gradient ${\nabla }_{x}H\left(x\mathrm{,}y\right)$ is globally Lipschitz with constant $L_3\left(y\right)$, which implies

$\Vert {\nabla }_{x}H\left(x_1\mathrm{,}y\right)-{\nabla }_{x}H\left(x_2\mathrm{,}y\right)\Vert \le L_3\left(y\right)\Vert x_1-x_2\Vert \mathrm{,}\forall x_1\mathrm{,}{x}_2\in R^n\mathrm{.}$

By Lemma 2.5, we know

$H\left(x_{k+1}\mathrm{,}{y}_k\right)-H\left(x_k\mathrm{,}{y}_k\right)\le -\langle {\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_k\right)\mathrm{,}{x}_k-x_{k+1}\rangle +\frac{L_3\left(y_k\right)}{2}{\Vert x_{k+1}-x_k\Vert }^2\mathrm{.}$ (31)

Besides, $f$ is semiconvex with parameter $\omega$, which implies (2.4) as follows:

$f\left(x_{k+1}\right)-f\left(x_k\right)\le -\langle p\mathrm{,}{x}_k-x_{k+1}\rangle +\frac{\omega }{2}{\Vert x_{k+1}-x_k\Vert }^2\mathrm{,}$

for all $x,y\in {ℛ}^n$ and all $p\in \partial f\left(x_{k+1}\right)$.

From optimality condition (13), we could know

$-{\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_k\right)-A^{\text{T}}{\lambda }_k+{\beta }_1{A}^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right)\in \partial f\left(x_{k+1}\right)\mathrm{.}$

Therefore,

$\begin{array}{l}f\left(x_{k+1}\right)-f\left(x_k\right)\\ \le \langle -{\nabla }_{x}H\left(x_{k+1},y_k\right)-A^{\text{T}}{\lambda }_k+{\beta }_1{A}^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right),x_{k+1}-x_k\rangle \\ +\frac{\omega }{2}{\Vert x_{k+1}-x_k\Vert }^2\\ =\langle -{\nabla }_{x}H\left(x_{k+1},y_k\right),x_{k+1}-x_k\rangle -\langle A^{\text{T}}{\lambda }_k,x_{k+1}-x_k\rangle \\ +\langle {\beta }_1{A}^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right),x_{k+1}-x_k\rangle +\frac{\omega }{2}{\Vert x_{k+1}-x_k\Vert }^2.\end{array}$ (32)

Inserting Equation (32), (31) into (30) yields

$\begin{array}{l}ℒ\left(x_{k+1}\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)-ℒ\left(x_k\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)\\ \le \langle -{\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_k\right)\mathrm{,}{x}_{k+1}-x_k\rangle -\langle A^{\text{T}}{\lambda }_k\mathrm{,}{x}_{k+1}-x_k\rangle \\ +\langle {\beta }_1{A}^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right)\mathrm{,}{x}_{k+1}-x_k\rangle +\frac{\omega }{2}{\Vert x_{k+1}-x_k\Vert }^2\\ -\langle {\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_k\right)\mathrm{,}{x}_k-x_{k+1}\rangle +\frac{L_3\left(y_k\right)}{2}{\Vert x_{k+1}-x_k\Vert }^2\\ +\langle {\lambda }_k\mathrm{,}A{x}_{k+1}-A{x}_k\rangle +{\beta }_2{D}_h\left(-y_k\mathrm{,}A{x}_{k+1}-b\right)-{\beta }_2{D}_h\left(-y_k\mathrm{,}A{x}_k-b\right)\end{array}$

$\begin{array}{l}+\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_k-b\Vert }^2-\frac{{\beta }_1}{2}{\Vert A{x}_k+y_k-b\Vert }^2\\ =-{\beta }_1\langle A^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right)\mathrm{,}{x}_k-x_{k+1}\rangle +\frac{\omega }{2}{\Vert x_{k+1}-x_k\Vert }^2+\frac{L_3}{2}{\Vert x_{k+1}-x_k\Vert }^2\\ +{\beta }_2{D}_h\left(-y_k\mathrm{,}A{x}_{k+1}-b\right)-{\beta }_2{D}_h\left(-y_k\mathrm{,}A{x}_k-b\right)+\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_k-b\Vert }^2\\ -\frac{{\beta }_1}{2}{\Vert A{x}_k+y_k-b\Vert }^2\mathrm{.}\end{array}$ (33)

From the three-point equation, we get

$\begin{array}{l}{\beta }_2{D}_h\left(-y_k,A{x}_{k+1}-b\right)-{\beta }_2{D}_h\left(-y_k,A{x}_k-b\right)\\ =-{\beta }_2(D_h\left(-y_k,A{x}_k-b\right)-D_h\left(-y_k,A{x}_{k+1}-b\right)\\ -D_h\left(A{x}_{k+1}-b,A{x}_k-b\right))-{\beta }_2{D}_h\left(A{x}_{k+1}-b,A{x}_k-b\right)\\ =-{\beta }_2\langle \nabla h\left(A{x}_k-b\right)-\nabla h\left(A{x}_{k+1}-b\right),A{x}_{k+1}+y_k-b\rangle \\ -{\beta }_2{D}_h\left(A{x}_{k+1}-b,A{x}_k-b\right).\end{array}$ (34)

Using (28), we could know

$-D_h\left(A{x}_{k+1}-b\mathrm{,}A{x}_k-b\right)\le -\frac{{\mu }_h}{2}{\Vert A{x}_{k+1}-A{x}_k\Vert }^2\mathrm{.}$ (35)

Substituting above two equations (34), (35) into (33), we obtain

$\begin{array}{l}ℒ\left(x_{k+1}\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)-ℒ\left(x_k\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)\\ \le \frac{\omega +L_3}{2}{\Vert x_{k+1}-x_k\Vert }^2-\frac{{\beta }_2{\mu }_h}{2}{\Vert A{x}_{k+1}-A{x}_k\Vert }^2+\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_k-b\Vert }^2\\ -\frac{{\beta }_1}{2}{\Vert A{x}_k+y_k-b\Vert }^2-{\beta }_1\langle A^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right)\mathrm{,}{x}_k-x_{k+1}\rangle \\ -{\beta }_2\langle \nabla h\left(A{x}_k-b\right)-\nabla h\left(A{x}_{k+1}-b\right)\mathrm{,}A{x}_{k+1}+y_k-b\rangle \mathrm{.}\end{array}$ (36)

By the fact that $\nabla h$ is Lipschitz continuous with $L_h$ on any bounded subset of ${ℝ}^m$, we get

$\begin{array}{l}-{\beta }_2\langle \nabla h\left(A{x}_k-b\right)-\nabla h\left(A{x}_{k+1}-b\right)\mathrm{,}A{x}_{k+1}+y_k-b\rangle \\ \le {\beta }_2{L}_h\Vert A{x}_{k+1}+y_k-b\Vert \Vert A{x}_k-A{x}_{k+1}\Vert \mathrm{.}\end{array}$ (37)

Using Equation (21), (29) and (36), which yields

$\begin{array}{l}ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_{k+1}\right)-ℒ\left(x_k\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)\\ \le \frac{\omega +L_3}{2}{\Vert x_{k+1}-x_k\Vert }^2-\frac{{\beta }_2{\mu }_h}{2}{\Vert A{x}_{k+1}-A{x}_k\Vert }^2-\frac{{\beta }_1}{2}{\Vert A{x}_k+y_k-b\Vert }^2\\ +\frac{L{L}_h+L_2-{\beta }_2{\mu }_h}{2}{\Vert y_{k+1}-y_k\Vert }^2+\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_{k+1}-b\Vert }^2+\frac{1}{{\beta }_2{\mu }_h}{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2\\ -{\beta }_1\langle A^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right)\mathrm{,}{x}_k-x_{k+1}\rangle +{\beta }_2{L}_h\Vert A{x}_{k+1}+y_k-b\Vert \Vert A{x}_k-A{x}_{k+1}\Vert \mathrm{.}\end{array}$ (38)

It is not difficult to see

$\begin{array}{l}-\frac{{\beta }_1}{2}{\Vert A{x}_k+y_k-b\Vert }^2-{\beta }_1\langle A^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right),x_k-x_{k+1}\rangle \\ =-\frac{{\beta }_1}{2}{\Vert A{x}_{k+1}+y_k-b+A{x}_k-A{x}_{k+1}\Vert }^2-{\beta }_1\langle A^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right),x_k-x_{k+1}\rangle \\ =-\frac{{\beta }_1}{2}\left({\Vert A{x}_{k+1}+y_k-b\Vert }^2+{\Vert A{x}_k-A{x}_{k+1}\Vert }^2+2\langle A{x}_{k+1}+y_k-b,A{x}_k-A{x}_{k+1}\rangle \right)\\ -{\beta }_1\langle A^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right),x_k-x_{k+1}\rangle \end{array}$

$\begin{array}{l}=-\frac{{\beta }_1}{2}\left({\Vert A{x}_{k+1}+y_k-b\Vert }^2+{\Vert A{x}_k-A{x}_{k+1}\Vert }^2\right)\\ \le -{\beta }_1\Vert A{x}_{k+1}+y_k-b\Vert \Vert A{x}_k-A{x}_{k+1}\Vert \\ =-{\beta }_2{L}_h\Vert A{x}_{k+1}+y_k-b\Vert \Vert A{x}_k-A{x}_{k+1}\Vert \left({\beta }_1=L_h{\beta }_2\right).\end{array}$ (39)

Using the condition that ${\beta }_1=L_h{\beta }_2$, which yields

$-\frac{{\beta }_1}{2}{\Vert A{x}_k+y_k-b\Vert }^2-{\beta }_1\langle A^{\text{T}}\left(-A{x}_{k+1}-y_k+b\right),x_k-x_{k+1}\rangle$

$-{\beta }_2\langle \nabla h\left(A{x}_k-b\right)-\nabla h\left(A{x}_{k+1}-b\right)\mathrm{,}A{x}_{k+1}+y_k-b\rangle \le 0$ (40)

From the assumption that $A^{\text{T}}A\succcurlyeq \mu I$ for some $\mu >0$, we obtain

${\Vert A{x}_{k+1}-A{x}_k\Vert }^2\ge \mu {\Vert x_{k+1}-x_k\Vert }^2\mathrm{.}$ (41)

Substituting (40), (41) and (20) into (38), which yields

$\begin{array}{l}ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_{k+1}\right)-ℒ\left(x_k\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)\\ \le \frac{L{L}_h+L_2-{\beta }_2{\mu }_h}{2}{\Vert y_{k+1}-y_k\Vert }^2+\frac{\omega +L_3}{2}{\Vert x_{k+1}-x_k\Vert }^2\\ -\frac{{\beta }_2{\mu }_h}{2}{\Vert A{x}_{k+1}-A{x}_k\Vert }^2+\frac{{\beta }_1+2{\beta }_2{\mu }_h}{2{\beta }_2{}^2{\mu }_h^2}{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2\\ \le \frac{L{L}_h+L_2-{\beta }_2{\mu }_h}{2}{\Vert y_{k+1}-y_k\Vert }^2+\frac{\omega +L_3-{\beta }_2{\mu }_h\mu }{2}{\Vert x_{k+1}-x_k\Vert }^2\\ +\frac{L_h+2{\mu }_h}{2{\beta }_2{\mu }_h^2}{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2\mathrm{.}\end{array}$ (42)

From the optimality condition (13), we have

${\lambda }_{k+1}=-\left(\nabla g\left(y_{k+1}\right)+{\nabla }_{y}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)\right)\mathrm{.}$ (43)

Then,

$\begin{array}{c}{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2={\Vert \nabla g\left(y_k\right)-\nabla g\left(y_{k+1}\right)+{\nabla }_{y}H\left(x_k\mathrm{,}{y}_k\right)-{\nabla }_{y}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)\Vert }^2\\ \le 2{\Vert \nabla g\left(y_k\right)-\nabla g\left(y_{k+1}\right)\Vert }^2+2{\Vert {\nabla }_{y}H\left(x_k\mathrm{,}{y}_k\right)-{\nabla }_{y}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)\Vert }^2\mathrm{.}\end{array}$ (44)

Using assumption 4),

$\begin{array}{l}\Vert \left({\nabla }_{x}H\left(x_1\mathrm{,}{y}_1\right)-{\nabla }_{x}H\left(x_2\mathrm{,}{y}_2\right)\mathrm{,}{\nabla }_{y}H\left(x_1\mathrm{,}{y}_1\right)-{\nabla }_{y}H\left(x_2\mathrm{,}{y}_2\right)\right)\Vert \\ \le M\Vert \left(x_1-x_2\mathrm{,}{y}_1-y_2\right)\Vert \end{array}$

We can obtain

$\begin{array}{l}{\Vert {\nabla }_{x}H\left(x_k\mathrm{,}{y}_k\right)-{\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)\Vert }^2+{\Vert {\nabla }_{y}H\left(x_k\mathrm{,}{y}_k\right)-{\nabla }_{y}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)\Vert }^2\\ \le M^2{\Vert x_{k+1}-x_k\Vert }^2+M^2{\Vert y_{k+1}-y_k\Vert }^2\mathrm{,}\end{array}$

which implies

$2{\Vert {\nabla }_{y}H\left(x_k\mathrm{,}{y}_k\right)-{\nabla }_{y}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)\Vert }^2\le 2M^2{\Vert x_{k+1}-x_k\Vert }^2+2M^2{\Vert y_{k+1}-y_k\Vert }^2\mathrm{.}$ (45)

Next, we consider two different scenarios:

1) $y_{k+1}\ne y_k$,

From the given assumption that $\left(g\mathrm{,}h\right)$ is L-smooth adaptable and Equation (9), which yields

$\Vert \nabla g\left(x\right)-\nabla g\left(y\right)\Vert \le L\frac{D_h\left(x\mathrm{,}y\right)+D_h\left(y\mathrm{,}x\right)}{\Vert x-y\Vert }$, for all $x\ne y\in int\left(dom\left(h\right)\right)$,

then,

$\begin{array}{l}\Vert \nabla g\left(y_{k+1}\right)-\nabla g\left(y_k\right)\Vert \\ \le L\frac{D_h\left(y_{k+1}\mathrm{,}{y}_k\right)+D_h\left(y_k\mathrm{,}{y}_{k+1}\right)}{\Vert y_{k+1}-y_k\Vert }\\ =L\frac{h\left(y_{k+1}\right)-h\left(y_k\right)-\langle \nabla h\left(y_k\right)\mathrm{,}{y}_{k+1}-y_k\rangle +h\left(y_k\right)-h\left(y_{k+1}\right)-\langle \nabla h\left(y_{k+1}\right)\mathrm{,}{y}_k-y_{k+1}\rangle }{\Vert y_{k+1}-y_k\Vert }\\ =L\frac{\langle \nabla h\left(y_k\right)-\nabla h\left(y_{k+1}\right)\mathrm{,}{y}_k-y_{k+1}\rangle }{\Vert y_{k+1}-y_k\Vert }\\ \le L\frac{L_h{\Vert y_k-y_{k+1}\Vert }^2}{\Vert y_{k+1}-y_k\Vert }=L{L}_h\Vert y_{k+1}-y_k\Vert \mathrm{.}\end{array}$ (46)

2) $y_{k+1}=y_k$, it is easy to verify that (46) holds.

By substituting Equation (45) and (46) into (44), we obtain

$\begin{array}{c}{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2\le 2L^2{L}_h^2{\Vert y_{k+1}-y_k\Vert }^2+2M^2{\Vert x_{k+1}-x_k\Vert }^2+2M^2{\Vert y_{k+1}-y_k\Vert }^2\\ =\left(2L^2{L}_h^2+2M^2\right){\Vert y_{k+1}-y_k\Vert }^2+2M^2{\Vert x_{k+1}-x_k\Vert }^2\mathrm{.}\end{array}$ (47)

Combining (47) and (42), we have

$\begin{array}{l}ℒ\left(x_{k+1}\mathrm{,}{y}_{k+1}\mathrm{,}{\lambda }_{k+1}\right)-ℒ\left(x_k\mathrm{,}{y}_k\mathrm{,}{\lambda }_k\right)\\ \le \frac{{\beta }_2{\mu }_h^2\left(L{L}_h+L_2-{\beta }_2{\mu }_h\right)+\left(L_h+2{\mu }_h\right)\left(2L^2{L}_h^2+2M^2\right)}{2{\beta }_2{\mu }_h^2}{\Vert y_{k+1}-y_k\Vert }^2\\ +\frac{{\beta }_2{\mu }_h^2\left(\omega +L_3-{\beta }_2{\mu }_h\mu \right)+2M^2\left(L_h+2{\mu }_h\right)}{2{\beta }_2{\mu }_h^2}{\Vert x_{k+1}-x_k\Vert }^2\mathrm{.}\end{array}$ (48)

By simple calculation, when

${\beta }_2>\max \left\{{\beta }_x,{\beta }_y\right\}$

where

${\beta }_y\mathrm{<mo>\; :\; </mo>}=\frac{L{L}_h+L_2+\sqrt{{\left(L{L}_h+L_2\right)}^2+4{\mu }_h^3\left(L_h+2{\mu }_h\right)\left(2L^2{L}_h^2+2M^2\right)}}{2{\mu }_h^3}\mathrm{,}$ (49)

${\beta }_x\mathrm{<mo>\; :\; </mo>}=\frac{\omega +L_3+\sqrt{{\left(\omega +L_3\right)}^2+8{\mu }_h^3\mu M^2\left(L_h+2{\mu }_h\right)}}{2{\mu }_h^3\mu }\mathrm{,}$ (50)

then

$ℒ\left(w_{k+1}\right)\le ℒ\left(w_k\right)-\delta {\Vert v_{k+1}-v_k\Vert }^2\mathrm{,}$

where

$\delta \mathrm{<mo>\; :\; </mo>}=\min \left\{{\delta }_1\mathrm{,}{\delta }_2\right\}$

${\delta }_1\mathrm{<mo>\; :\; </mo>}=\frac{-L{L}_h-L_2+\beta {\mu }_h}{2}-\frac{2L^2{L}_h^2+2M^2}{{\beta }_2{\mu }_h}-\frac{L^2{L}_h^3+M^2}{{\beta }_2{\mu }_h^2}$

${\delta }_2\mathrm{<mo>\; :\; </mo>}=\frac{-\omega -L_3+{\beta }_2{\mu }_h\mu }{2}-\frac{M^2\left(L_h+2{\mu }_h\right)}{{\beta }_2{\mu }_h^2}\mathrm{.}$

Thus, the theorem is established.

Remark 3.1. If $H=0$, then we have $L_2=L_3=M=0$. Moreover, by

$L_h={\mu }_h=1$, we have ${\beta }_2>\max \left\{3L,\mathrm{<mi>\; </mi>}\frac{\omega }{\mu }\right\}$. This result is larger than 2L which is

proved by Guo et al.’s classical result [8] . Howerver, if we directly solve the separable problem, it can cover their conclusion. The reason of our weaker conclusion could be that the inequality bounds are not tight enough in our proof.

Lemma 3.2. Let ${\left\{{\omega }_k\right\}}_{k\in N}$ be the sequence generated by the Bragman ADMM (6) which is assumed to be bounded. Then the following holds

${\displaystyle \underset{k=0}{\overset{+\infty }{\sum }}}{\Vert {\omega }_{k+1}-{\omega }_k\Vert }^2<+\infty .$ (51)

Proof.

Since $\left\{{\omega }_k\right\}$ is bounded, then there exists a subsequence $\left\{{\omega }_{k_j}\right\}$ such that ${\omega }_{k_j}\to {\omega }^{\mathrm{<mo>\; *\; </mo>}}$. By the condition that f is lower semi-continuous and g, H are continuous, we obtain $L(\cdot )$ is semicontinuous, which implies

$ℒ\left({\omega }^{\mathrm{<mo>\; *\; </mo>}}\right)\le \underset{j\to +\infty }{lim}\inf ℒ\left({\omega }_{k_j}\right)\mathrm{.}$

Consequently, ${\left\{ℒ\left({\omega }_{k_j}\right)\right\}}_{j\in N}$ is bounded from below.

Note that, Lemma 3.1 implies that ${\left\{ℒ\left({\omega }_{k_j}\right)\right\}}_{j\in N}$ is nonincreasing and thus ${\left\{ℒ\left({\omega }_{k_j}\right)\right\}}_{j\in N}$ is convergent. Moreover, we have ${\left\{ℒ\left({\omega }_{k_j}\right)\right\}}_{j\in N}$ is convergent and $ℒ\left({\omega }_k\right)\ge ℒ\left({\omega }^{\mathrm{<mo>\; *\; </mo>}}\right)$. Reassanging terms of (3.1) leads to

$\delta {\Vert v_{k+1}-v_k\Vert }^2\le ℒ\left({\omega }_k\right)-ℒ\left({\omega }_{k+1}\right)\mathrm{.}$

Now, summing the above inequality over $k=\mathrm{0,}\cdots \mathrm{,}n$ yields

${\displaystyle \underset{k=0}{\overset{n}{\sum }}}\delta {\Vert v_{k+1}-v_k\Vert }^2\le ℒ\left({\omega }_0\right)-ℒ\left({\omega }_{n+1}\right)\le ℒ\left({\omega }_0\right)-ℒ\left({\omega }^{*}\right)<+\infty .$

Since $\delta >0$, we have ${\displaystyle {\sum }_{k=0}^{+\infty }}{\Vert v_{k+1}-v_k\Vert }^2<+\infty$, thus

${\displaystyle \underset{k=0}{\overset{+\infty }{\sum }}}{\Vert x_{k+1}-x_k\Vert }^2<+\infty ,{\displaystyle \underset{k=0}{\overset{+\infty }{\sum }}}{\Vert y_{k+1}-y_k\Vert }^2<+\infty .$ (52)

Moreover, it follows from (52) and (47) that ${\displaystyle {\sum }_{k=0}^{+\infty }}{\Vert {\lambda }_{k+1}-{\lambda }_k\Vert }^2<+\infty$, therefore, we obtain ${\displaystyle {\sum }_{k=0}^{+\infty }}{\Vert {\omega }_{k+1}-{\omega }_k\Vert }^2<+\infty$.

Lemma 3.3. Let ${\left\{{\omega }_k\right\}}_{k\in N}$ be the sequence generated by the Bragman ADMM (6) which is assumed to be bounded. Then there exists $\zeta >0$ such that

$d\left(\mathrm{0,}\partial ℒ\left({\omega }_{k+1}\right)\right)\le \zeta \Vert v_{k+1}-v_k\Vert \mathrm{.}$ (53)

Proof.

By the definition of function $ℒ(\cdot )$, it follows

$(\begin{array}{l}{\partial }_xℒ\left({\omega }_{k+1}\right)=\partial f\left(x_{k+1}\right)+{\nabla }_{x}H\left(x_{k+1},y_{k+1}\right)+A^{\text{T}}{\lambda }_{k+1}+L_h{\beta }_2{A}^{\text{T}}\langle A{x}_{k+1}+y_{k+1}-b\rangle \\ +{\beta }_2{A}^{\text{T}}\langle \nabla h^2\left(A{x}_{k+1}-b\right),A{x}_{k+1}+y_{k+1}-b\rangle \\ {\partial }_yℒ\left({\omega }_{k+1}\right)=\nabla g\left(y_{k+1}\right)+{\nabla }_{y}H\left(x_{k+1},y_{k+1}\right)+{\lambda }_{k+1}+L_h{\beta }_2\left(A{x}_{k+1}+y_{k+1}-b\right)\\ +{\beta }_2\left(\nabla h\left(A{x}_{k+1}-b\right)-\nabla h\left(-y_{k+1}\right)\right)\\ {\partial }_{\lambda }ℒ\left({\omega }_{k+1}\right)=A{x}_{k+1}+y_{k+1}-b.\end{array}$(54)

This, together with Equation (13), which yields

$(\begin{array}{l}{\partial }_xℒ\left({\omega }_{k+1}\right)={\nabla }_{x}H\left(x_{k+1},y_{k+1}\right)-{\nabla }_{x}H\left(x_{k+1},y_k\right)+A^{\text{T}}\left({\lambda }_{k+1}-{\lambda }_k\right)\\ +L_h{\beta }_2{A}^{\text{T}}\langle y_{k+1}-y_k\rangle +{\beta }_2{A}^{\text{T}}\langle \nabla h^2\left(A{x}_{k+1}-b\right),A{x}_{k+1}+y_{k+1}-b\rangle \\ {\partial }_yℒ\left({\omega }_{k+1}\right)={\lambda }_{k+1}-{\lambda }_k+L_h{\beta }_2\left(A{x}_{k+1}+y_{k+1}-b\right)\\ {\partial }_{\lambda }ℒ\left({\omega }_{k+1}\right)=A{x}_{k+1}+y_{k+1}-b.\end{array}$ (55)

Let

$\begin{array}{l}\left({\xi }_{k+1}^1\mathrm{,}{\xi }_{k+1}^2\mathrm{,}{\xi }_{k+1}^3\right)\\ \mathrm{<mo>\; :\; </mo>}=({\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)-{\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_k\right)+A^{\text{T}}\left({\lambda }_{k+1}-{\lambda }_k\right)+L_h{\beta }_2{A}^{\text{T}}\langle y_{k+1}-y_k\rangle \\ +{\beta }_2{A}^{\text{T}}\langle \nabla h^2\left(A{x}_{k+1}-b\right)\mathrm{,}A{x}_{k+1}+y_{k+1}-b\rangle \mathrm{,}{\lambda }_{k+1}-{\lambda }_k\\ +L_h{\beta }_2\left(A{x}_{k+1}+y_{k+1}-b\right)\mathrm{,}A{x}_{k+1}+y_{k+1}-b)\mathrm{.}\end{array}$ (56)

Then it follows from Lemma 2.3 that $\left({\xi }_{k+1}^1\mathrm{,}{\xi }_{k+1}^2\mathrm{,}{\xi }_{k+1}^3\right)\in \partial ℒ\left({\omega }_{k+1}\right)$. Furthermore, there exist ${\zeta }_1,{\zeta }_2,{\zeta }_3>0$ such that

$\begin{array}{c}\Vert \left({\xi }_{k+1}^1\mathrm{,}{\xi }_{k+1}^2\mathrm{,}{\xi }_{k+1}^3\right)\Vert \le {\zeta }_1\Vert y_{k+1}-y_k\Vert +{\zeta }_2\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \\ +{\zeta }_3\Vert {\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)-{\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_k\right)\Vert \mathrm{.}\end{array}$ (57)

Again, since $\nabla H$ is lipschitz continuous on bounded subsets and $\left\{\left(x_k\mathrm{,}{y}_k\right)\right\}$ is bounded, it follows

$\Vert {\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_{k+1}\right)-{\nabla }_{x}H\left(x_{k+1}\mathrm{,}{y}_k\right)\Vert \le M\Vert y_{k+1}-y_k\Vert \mathrm{.}$ (58)

From (47), we know

$\Vert {\lambda }_{k+1}-{\lambda }_k\Vert \le \sqrt{2L^2{L}_h^2+2M^2}\Vert y_{k+1}-y_k\Vert +\sqrt{2}M\Vert x_{k+1}-x_k\Vert \mathrm{.}$ (59)

By setting $\zeta =\sqrt{{\left({\zeta }_1+\sqrt{2L^2{L}_h^2+2M^2}{\zeta }_2+M{\zeta }_3\right)}^2+{\left(\sqrt{2}M{\zeta }_3\right)}^2}$, it follows above three Equations (57), (58), (59), we can obtain

$\begin{array}{c}d\left(\mathrm{0,}\partial ℒ\left({\omega }_{k+1}\right)\right)\le \Vert \left({\xi }_{k+1}^1\mathrm{,}{\xi }_{k+1}^2\mathrm{,}{\xi }_{k+1}^3\right)\Vert \\ \le \left({\zeta }_1+\sqrt{2L^2{L}_h^2+2M^2}{\zeta }_2+M{\zeta }_3\right)\Vert y_{k+1}-y_k\Vert \\ +\sqrt{2}M{\zeta }_3\Vert x_{k+1}-x_k\Vert \\ \le \zeta \Vert v_{k+1}-v_k\Vert \mathrm{,}\end{array}$ (60)

where the third inequality follows from the Cauchy inequality. Thus, the Lemma 3.3 is proven.

Lemma 3.4. Let ${\left\{{\omega }_k\right\}}_{k\in N}$ be the sequence generated by the Bragman ADMM (6) which is assumed to be bounded. Let $S\left({\omega }_0\right)$ denote the set of its limit point. Then,

1) $S\left({\omega }_0\right)$ is a nonempty compact set,and $d\left({\omega }_k\mathrm{,}S\left({\omega }_0\right)\to 0\right)$, as $k\to +\infty$;

2) $S\left({\omega }_0\right)\subseteq crit{ℒ}_{\beta }$;

3) $ℒ(\cdot )$ is finite and constant on $S\left({\omega }_0\right)$, equal to ${\inf }_{k\in N}ℒ\left({\omega }_k\right)={\lim }_{k\to +\infty }ℒ\left({\omega }_k\right)$.

Proof.

1) This item follows as an elementary consequence of the definition of limit points.

2) For any fixed $\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)\in S\left(w^0\right)$, there exists a subsequence ${\left\{\left(x^{k_j}\mathrm{,}{y}^{k_j}\mathrm{,}{\lambda }^{k_j}\right)\right\}}_{j\in N}$ of ${\left\{\left(x^k\mathrm{,}{y}^k\mathrm{,}{\lambda }^k\right)\right\}}_{k\in N}$ converges to $\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)$.

Since $x^{k+1}$ is the global minimizer of $ℒ\left(x\mathrm{,}{y}^k\mathrm{,}{\lambda }^k\right)$ for the variable $x$, it holds

$ℒ\left(x^{k+1}\mathrm{,}{y}^k\mathrm{,}{\lambda }^k\right)\le ℒ\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^k\mathrm{,}{\lambda }^k\right)\mathrm{.}$

Using the above inequality and the continuity of $ℒ(\cdot )$ with respect to $y$ and $\lambda$ ensure

$\underset{j\to +\infty }{\lim \sup }{ℒ}_{\beta }\left(x^{k_j+1}\mathrm{,}{y}^{k_j}\mathrm{,}{\lambda }^{k_j}\right)=\underset{j\to +\infty }{\lim \sup }ℒ\left(x^{k_j+1}\mathrm{,}{y}^{k_j+1}\mathrm{,}{\lambda }^{k_j+1}\right)\le ℒ\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)\mathrm{.}$

On the other hand, Lemma 3.3 implies $\Vert w^{k+1}-w^k\Vert \to 0$, which means that the subsequence ${\left\{\left(x^{k_j+1}\mathrm{,}{y}^{k_j+1}\mathrm{,}{\lambda }^{k_j+1}\right)\right\}}_{j\in N}$ also converges to $\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)$.

From the lower semicontinuity of $ℒ(\cdot )$, we have

$\underset{j\to +\infty }{\lim \inf }ℒ\left(x^{k_j+1},y^{k_j+1},{\lambda }^{k_j+1}\right)\ge ℒ\left(x^{*},y^{*},{\lambda }^{*}\right).$ (61)

Then by combining above two equations together we can get

$\underset{j\to +\infty }{\lim }ℒ\left(x^{k_j+1},y^{k_j+1},{\lambda }^{k_j+1}\right)=ℒ\left(x^{*},y^{*},{\lambda }^{*}\right),$ (62)

which implies

$\underset{j\to +\infty }{\lim }f\left(x^{k_j+1}\right)=f\left(x^{*}\right).$ (63)

Passing to the limit in (13) and (54) along the subsequence ${\left\{\left(x^{k_j+1},y^{k_j+1},{\lambda }^{k_j+1}\right)\right\}}_{j\in N}$ and invoking (63), the continuity of $\nabla g(\cdot )$, ${\nabla }_{x}H\left(\cdot \mathrm{,}\cdot \right)$, ${\nabla }_{y}H\left(\cdot \mathrm{,}\cdot \right)$, it follows that

$(\begin{array}{l}-{\nabla }_{x}H\left(x^{*},y^{*}\right)-A^{\text{T}}{\lambda }^{*}\in \partial f\left(x^{*}\right)\\ -{\nabla }_{y}H\left(x^{*},y^{*}\right)-{\lambda }^{*}=\nabla g\left(y^{*}\right)\\ \nabla h\left(A{x}^{*}-b\right)-\nabla h\left(-y^{*}\right)=0.\end{array}$

The last equation implies that $A{x}^{*}+y^{*}-b=0$ due to the strong convexity of $h(\cdot )$.

By the definition of $ℒ(\cdot )$, we have

$(\begin{array}{l}0\in {\partial }_xℒ\left({\omega }^{*}\right)=\partial f\left(x^{*}\right)+{\nabla }_{x}H\left(x^{*},y^{*}\right)+A^{\text{T}}{\lambda }^{*}\\ 0\in {\partial }_yℒ\left({\omega }^{*}\right)=\nabla g\left(y^{*}\right)+{\nabla }_{y}H\left(x^{*},y^{*}\right)+{\lambda }^{*}\\ 0\in {\partial }_{\lambda }ℒ\left({\omega }^{*}\right)=A{x}^{*}+y^{*}-b.\end{array}$ (64)

By the definition of critical point of the augmented Lagrangian function ${ℒ}_{\beta }(\cdot )$ (1.2), if it satisfies:

$\{\begin{array}{l}{A}^{\text{T}}{\lambda }^{*}-{\nabla }_{x}H\left(x^{*},y^{*}\right)\in \partial f\left(x^{*}\right)\\ {\lambda }^{*}-{\nabla }_{y}H\left(x^{*},y^{*}\right)=\nabla g\left(y^{*}\right)\\ A{x}^{*}+y^{*}-b=0\end{array}$

Then, $\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)$ is a critical point of ${ℒ}_{\beta }\mathrm{,}ℒ$, i.e., $\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)\in \text{crit}{ℒ}_{\beta }$, and $\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)\in \text{crit}ℒ$.

3) For any point $\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)\in S\left(w^0\right)$, there exists a subsequence ${\left\{\left(x^{k_j}\mathrm{,}{y}^{k_j}\mathrm{,}{\lambda }^{k_j}\right)\right\}}_{j\in N}$ converges to $\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)$.

Since ${\left\{ℒ\left(w^k\right)\right\}}_{k\in N}$ is nonincreasing, combining (61) and (62) leads to

$\underset{k\to +\infty }{\lim }ℒ\left(x^k\mathrm{,}{y}^k\mathrm{,}{\lambda }^k\right)=ℒ\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)\mathrm{.}$ (65)

Therefore, $ℒ(\cdot )$ is constant on $S\left(w^0\right)$. Moreover, ${\inf }_{k\in N}ℒ\left(w^k\right)={\lim }_{k\to +\infty }ℒ\left(w^k\right)$.

Next, we will prove the convergence of the iterative sequence.

Theorem 3.1. Let ${\left\{w^k\right\}}_{k\in N}$ be the sequence generated by the ADMM (6) which is assumed to be bounded. Suppose that $ℒ(\cdot )$ is a KL function, then ${\left\{w^k\right\}}_{k\in N}$ has finite length, that is

${\displaystyle \underset{k=0}{\overset{+\infty }{\sum }}}\Vert w^{k+1}-w^k\Vert <+\infty ,$

and as a consequence, we have ${\left\{w^k\right\}}_{k\in N}$ converged to a critical point of ${ℒ}_{\beta }(\cdot )$.

Proof.

Since from the proof of Lemma (3.4), it follows that $ℒ\left(w^k\right)\to ℒ\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)$ for all $w^{\mathrm{<mo>\; *\; </mo>}}\in S\left(w^0\right)$. We consider two cases:

1) If there exists an integer $k_0$ for which $ℒ\left(w^{k_0}\right)=ℒ\left(w^{*}\right)$. Rearranging terms of (14) we have that for any $k>k_0$,

$\delta {\Vert v^{k+1}-v^k\Vert }^2\le ℒ\left(w^k\right)-ℒ\left(w^{k+1}\right)\le ℒ\left(w^{k_0}\right)-ℒ\left(w^{*}\right)=0,$

implying that $v^{k+1}=v^k$ for any $k>k_0$.

Associated with (47), for any $k>k_0+1$, it follows that $w^{k+1}=w^k$ and the assertion holds.

2) Now, assume $ℒ\left(w^k\right)>ℒ\left(w^{*}\right)$ for all $k$. We claim there exists $\tilde{k}>0$ such that for all $k>\tilde{k}$,

$\delta {\Vert v^{k+1}-v^k\Vert }^2\le \zeta \Vert v^k-v^{k-1}\Vert {\Delta }_{k\mathrm{,}k+1}\mathrm{,}$ (66)

where ${\Delta }_{p,q}:=\phi \left(ℒ\left(w^p\right)-ℒ\left(w^{*}\right)\right)-\phi \left(ℒ\left(w^q\right)-ℒ\left(w^{*}\right)\right)$. To see this, note that $d\left(w^k\mathrm{,}S\left(w^0\right)\right)\to 0$ and ${ℒ}_{\beta }\left(w^k\right)\to ℒ\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)$, then for all $\epsilon ,\eta >0$, there exists $\tilde{k}>0$ such that when $k>\tilde{k}$, we have

$d\left(w^k,S\left(w^0\right)\right)<\epsilon ,ℒ\left(w^{*}\right)<ℒ\left(w^k\right)<ℒ\left(w^{*}\right)+\eta .$

Since $S\left(w^0\right)$ is a nonempty compact set and $ℒ(\cdot )$ is constant on $S\left(w^0\right)$, applying Lemma (2.4) with $\Omega :=S\left(w^0\right)$, we deduce that for any $k>\tilde{k}$

${\phi }^{\prime }\left(ℒ\left(w^k\right)-ℒ\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)\right)d\left(\mathrm{0,}\partial ℒ\left(w^k\right)\right)\ge 1.$

Since $ℒ\left(w^k\right)-ℒ\left(w^{k+1}\right)=\left(ℒ\left(w^k\right)-ℒ\left(w^{*}\right)\right)-\left(ℒ\left(w^{k+1}\right)-{ℒ}_{\beta }\left(w^{*}\right)\right)$, making use of the concavity of $\phi$ we get that

$\begin{array}{l}\phi \left(ℒ\left(w^k\right)-ℒ\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)\right)-\phi \left({ℒ}_{\beta }\left(w^{k+1}\right)-{ℒ}_{\beta }\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)\right)\\ \ge {\phi }^{\prime }\left(ℒ\left(w^k\right)-ℒ\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)\right)\left({ℒ}_{\beta }\left(w^k\right)-{ℒ}_{\beta }\left(w^{k+1}\right)\right)\end{array}$

Thus, using the inequalities $d\left(\mathrm{0,}\partial ℒ\left(w^k\right)\right)\le \zeta \Vert v^k-v^{k-1}\Vert$ and ${\phi }^{\prime }\left(ℒ\left(w^k\right)-ℒ\left(w^{*}\right)\right)>0$, we obtain

$\begin{array}{l}ℒ\left(w^k\right)-ℒ\left(w^{k+1}\right)\\ \le \frac{\phi \left(ℒ\left(w^k\right)-ℒ\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)\right)-\phi \left(ℒ\left(w^{k+1}\right)-ℒ\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)\right)}{{\phi }^{\prime }\left(ℒ\left(w^k\right)-ℒ\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)\right)}\\ \le \zeta \Vert v^k-v^{k-1}\Vert \left[\phi \left(ℒ\left(w^k\right)-ℒ\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)\right)-\phi \left(ℒ\left(w^{k+1}\right)-ℒ\left(w^{\mathrm{<mo>\; *\; </mo>}}\right)\right)\right]\mathrm{.}\end{array}$

Combining Lemma 3.1 with the above relation gives (66), as desired. Moreover, (66) implies

$\Vert v^{k+1}-v^k\Vert \le \sqrt{\frac{\zeta }{\delta }{\Delta }_{k\mathrm{,}k+1}}{\Vert v^k-v^{k-1}\Vert }^{\frac{1}{2}}\mathrm{.}$

Notice that $2\sqrt{\alpha \beta }\le \alpha +\beta$, for all $\alpha \mathrm{,}\beta \ge 0$. Then we obtain

$2\Vert v^{k+1}-v^k\Vert \le \Vert v^k-v^{k-1}\Vert +\frac{\zeta }{\delta }{\Delta }_{k\mathrm{,}k+1}\mathrm{.}$

Summing the inequality above over $k=\tilde{k}+\mathrm{1,}\cdots \mathrm{,}m$ yields

$2{\displaystyle \underset{k=\tilde{k}+1}{\overset{m}{\sum }}}\Vert v^{k+1}-v^k\Vert \le {\displaystyle \underset{k=\tilde{k}+1}{\overset{m}{\sum }}}\Vert v^k-v^{k-1}\Vert +\frac{\zeta }{\delta }{\Delta }_{\tilde{k}+1,m+1}.$

Since $\phi \left(ℒ\left(w^{m+1}\right)-ℒ\left(w^{*}\right)\right)>0$, rearranging terms and letting $m\to +\infty$ lead to

${\displaystyle \underset{k=\tilde{k}+1}{\overset{+\infty }{\sum }}}\Vert v^{k+1}-v^k\Vert \le \Vert v^{\tilde{k}+1}-v^{\tilde{k}}\Vert +\frac{\zeta }{\delta }\phi \left(ℒ\left(w^{\tilde{k}+1}\right)-ℒ\left(w^{*}\right)\right),$

which implies

${\displaystyle \underset{k=0}{\overset{+\infty }{\sum }}}\Vert v^{k+1}-v^k\Vert <+\infty .$

Thus, it follows that

${\displaystyle \underset{k=0}{\overset{+\infty }{\sum }}}\Vert x^{k+1}-x^k\Vert <+\infty ,{\displaystyle \underset{k=0}{\overset{+\infty }{\sum }}}\Vert y^{k+1}-y^k\Vert <+\infty .$

These, together with (47), we obtain

${\displaystyle \underset{k=0}{\overset{+\infty }{\sum }}}\Vert {\lambda }^{k+1}-{\lambda }^k\Vert <+\infty .$

Moreover, note that

$\begin{array}{c}\Vert w^{k+1}-w^k\Vert ={\left({\Vert x^{k+1}-{\lambda }^k\Vert }^2+{\Vert y^{k+1}-y^k\Vert }^2+{\Vert {\lambda }^{k+1}-{\lambda }^k\Vert }^2\right)}^{\frac{1}{2}}\\ \le \Vert x^{k+1}-{\lambda }^k\Vert +\Vert y^{k+1}-y^k\Vert +\Vert {\lambda }^{k+1}-{\lambda }^k\Vert .\end{array}$

Therefore,

${\displaystyle \underset{k=0}{\overset{+\infty }{\sum }}}\Vert w^{k+1}-w^k\Vert <+\infty ,$

and ${\left\{w^k\right\}}_{k\in N}$ is a Cauchy sequence (see P.482 for a simple proof), which shows that the sequence generated by our algorithm converges. The assertion then follows immediately from Lemma 3.4.

We now establish the convergence rate for the ADMM (6). The proof of the following result is similar to that in [8] and hence omitted here.

Theorem 3.2 Let ${\left\{w^k\right\}}_{k\in N}$ be the sequence generated by the ADMM (6) and converges to $w^{*}:=\left(x^{*},y^{*},{\lambda }^{*}\right)$. Assume that $ℒ(\cdot )$ has the KL property at $\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)$ with $\phi \left(s\right):=c{s}^{1-\theta }$, $\theta \in \left[0,1\right)$, $c>0$. Then the following estimations hold:

1) If $\theta =0$, then the sequence ${\left\{w^k\right\}}_{k\in N}$ converges in a finite number of steps.

2) If $\theta \in \left(\mathrm{0,}\frac{1}{2}\right]$, then there exist $c>0$ and $\tau \in \left[0,1\right)$, such that

$\Vert \left(x^k\mathrm{,}{y}^k\mathrm{,}{\lambda }^k\right)-\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)\Vert \le c{\tau }^k\mathrm{.}$

3) If $\theta \in \left(\frac{1}{2}\mathrm{,1}\right)$, then there exists $c>0$, such that

$\Vert \left(x^k\mathrm{,}{y}^k\mathrm{,}{\lambda }^k\right)-\left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{y}^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}{\lambda }^{\mathrm{<mo>\; *\; </mo>}}\right)\Vert \le c{k}^{\frac{\theta -1}{2\theta -1}}\mathrm{.}$

4. Conclusions

In this paper, we conducted an analysis of the convergence properties of the Bregman Alternating Direction Method of Multipliers (ADMM) for addressing linearly constrained nonconvex minimization problems with coupled objective functions. Our study operated under the assumption that the associated functions satisfy the Kurdyka-Lojasiewicz inequality. We successfully demonstrated that the iterative sequence generated by the Bregman ADMM converges to a critical point of the augmented Lagrangian function, given that the penalty parameter in the augmented Lagrangian surpasses a certain threshold. Additionally, with further conditions on the problem’s data, we established the convergence rate of the algorithm. Looking forward, extending Algorithm (6) to handle multi-block non-separable linearly constrained nonconvex optimization problems and incorporating generalized ADMM represent meaningful avenues for future exploration.

Funding

These authors were supported by the Natural Science Foundation of China (Grant Nos. 11801455), the Sichuan Science and Technology Program (Grant No. 2023NSFSC1922), the Innovation Team Funds of China West Normal University (Grant No. KCXTD2023-3), the Fundamental Research Funds of China West Normal University (Grant No. 23kc010).

References

1. Alizadeh, M., Li, X., Wang, Z., Scaglione, A. and Melton, R. (2012) Demand-Side Management in the Smart Grid: Information Processing for the Power Switch. IEEE Signal Processing Magazine, 29, 55-67. >https://doi.org/10.1109/MSP.2012.2192951

2. Chang, T.H., Alizadeh, M. and Scaglione, A. (2012) Coordinated Home Energy Management for Real-Time Power Balancing. 2012 IEEE Power and Energy Society General Meeting, San Diego, 22-26 July 2012, 22-26.

3. Li, N., Chen, L. and Low, S.H. (2011) Optimal Demand Response Based on Utility Maximization in Power Networks. 2011 IEEE Power and Energy Society General Meeting, Detroit, 24-28 July 2011, 1-8. >https://doi.org/10.1109/PES.2011.6039082

4. Paatero, J.V. and Lund, P.D. (2006) A Model for Generating Household Electricity Load Profiles. International Journal of Energy Research, 30, 273-290. >https://doi.org/10.1002/er.1136

5. Scutari, G. and Palomar, D.P. (2010) MIMO Cognitive Radio: A Game Theoretical Approach. IEEE Transactions on Signal Processing, 58, 761-780. >https://doi.org/10.1109/TSP.2009.2032039

6. Zhao, Q. and Sadler, B.M. (2007) A Survey of Dynamic Spectrum Access. IEEE Signal Processing Magazine, 24, 79-89. >https://doi.org/10.1109/MSP.2007.361604

7. Guo, K., Han, D.R. and Wu, T.T. (2018) Convergence Analysis for Optimization Problems with Nonseparable Nonconvex Objective and Linear Constraints. Pacific Journal of Optimization, 14, 489-506.

8. Guo, K., Han, D.R. and Wu, T.T. (2017) Convergence of Alternating Method for Minimizing Sum of Two Nonconvex Functions with Linear Constraints. International Journal of Computer Mathematics, 94, 1653-1669. >https://doi.org/10.1080/00207160.2016.1227432

9. Kong, W.W. and Monteiro, R.D.C. (2024) Global Complexity Bound of a Proximal ADMM for Linearly-Constrained Nonseperable Nonconvex Composite Programming. SIAM Journal on Optimization, 34.

10. Guo, K., Han, D., Wang, D.Z.W. and Wu, T.T. (2017) Convergence of ADMM for Multi-Block Nonconvex Separable Optimization Models. Frontiers of Mathematics in China, 12, 1139-1162.

11. Guo, K. and Wang, X. (2018) Convergence of Generalized Alternating Direction Method of Multipliers for Nonseparable Nonconvex Objective with Linear Constraints. >https://doi.org/10.3770/j.issn:2095-2651.2018.05.010

12. Tan, L. and Guo, K. (2025) Bregman ADMM: A New Algorithm for Nonconvex Optimization with Linear Constraints. Journal of Nonlinear and Variational Analysis, 9, 179-196. >https://jnva.biemdas.com/archives/2622

13. Boyd, S., Parikh, N., Chu, E., Peleato, B. and Eckstein, J. (2010) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends in Machine Learning, 3, 1-122. >https://doi.org/10.1561/2200000016

14. Figueiredo, M.A.T. and Bioucas-Dias, J.M. (2010) Restoration of Poissonian Images Using Alternating Direction Optimization. IEEE Transactions on Image Processing, 19, 3133-3145. >https://doi.org/10.1109/TIP.2010.2053941

15. Goldstein, T., Bresson, X. and Osher, S. (2010) Geometric Applications of the Split Bregman Method: Segmentation and Surface Reconstruction. Journal of Scientific Computing, 45, 272-293. >https://doi.org/10.1007/s10915-009-9331-z

16. Fu, Q., Wang, H. and Banerjee, A. (2013) Bethe-ADMM for Tree Decomposition Based Parallel MAP Inference. arXiv: 1309.6829.

17. Wang, H. and Banerjee, A. (2013) Online alternating Direction Method. arXiv: 1206.6448.

18. Yang, J. and Zhang, Y. (2011) Alternating Direction Algorithms for ℓ ₁-Problems in Compressive Sensing. SIAM Journal on Scientific Computing, 33, 250-278. >https://doi.org/10.1137/090777761

19. Gabay, D. (1983) Applications of the Method of Multipliers to Variational Inequalities. Studies in Mathematics and Its Applications, 15, 299-331. >https://doi.org/10.1016/S0168-2024(08)70034-1

20. Kellogg, R.B. (1969) Nonlinear Alternating Direction Algorithm. Mathematics of Computation, 23, 23-38.

21. Marchuk, G.I. (1990) Splitting and Alternating Direction Methods. Handbook of Numerical Analysis, 1, 197-462. >https://doi.org/10.1016/S1570-8659(05)80035-3

22. Beck, A. (2017) First Order Methods In Optimization. SIAM, Philadelphia. >https://doi.org/10.1137/1.9781611974997

23. Bolte, J., Sabach, S., Teboulle, M. and Vaisbourd, T. (2018) First Order Methods beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems. SIAM Journal on Optimization, 28, 2131-2151. >https://doi.org/10.1137/17M1138558

24. Bauschke, H.H., Borwein, J. and Combettes, P.L. (2003) Bregman Monotone Optimization Algorithms. SIAM Journal on Control and Optimization, 42, 596-636. >https://doi.org/10.1137/S0363012902407120

25. Attouch, H., Bolte, J., Redont, P. and Soubeyran, A. (2010) Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka Lojasiewicz Inequality. Mathematics of Operations Research, 35, 438-457. >https://doi.org/10.1287/moor.1100.0449

26. Attouch, H., Bolte, J. and Svaiter, B.F. (2013) Convergence of Descent Methods for Semi-Algebraic and Tame Problems: Proximal Algorithms, Forward-Backward Splitting, and Regularized Guass-Seidel Methods. Mathematical Programming, 137, 91-129. >https://doi.org/10.1007/s10107-011-0484-9

27. Bolte, J., Sabach, S. and Teboulle, M. (2014) Proximal Alternating Linearized Minimization for Nonconvex and Nonsmooth Problem. Mathematical Programming, 146, 459-494. >https://doi.org/10.1007/s10107-013-0701-9

28. Nesterov, Y. (2004) Introduction Lectures on Convex Optimization: A Basic Course. Springer, New York. >https://doi.org/10.1007/978-1-4419-8853-9