11

Mixed $\mathscr{H}$₂/$\mathscr{H}$_∞Nonlinear Control

In this chapter, we discuss the mixed $\mathscr{H}$₂/$\mathscr{H}$_∞-control problem for nonlinear systems. This problem arises when a higher-degree of performance for the system is desired, and two criteria are minimized to derive the controller that enjoys both the properties of an $\mathscr{H}$₂ (or LQG [174]) and $\mathscr{H}$_∞-controller. A stronger motivation for this problem though is that, because the solution to the $\mathscr{H}$_∞-control problem is nonunique (if it is not optimal, it can hardly be unique) and only the suboptimal problem could be solved easily, then is it possible to formulate another problem that could be solved optimally and obtain a unique solution?

The problem for linear systems was first considered by Bernstein and Haddad [67], where a solution for the output-feedback problem in terms of three coupled algebraic-Riccati-equations (AREs) was obtained by formulating it as an LQG problem with an $\mathscr{H}$_∞ constraint. The dual to this problem has also been considered by Doyle, Zhou and Glover [93, 293]. While Mustapha and Glover [205, 206] have considered entropy minimization which provides an upper bound on the $\mathscr{H}$₂-cost under an $\mathscr{H}$_∞-constraint.

Another contribution to the linear literature was from Khargonekhar and Rotea [161] and Scherer et al. [238, 239], who considered more general multi-objective problems using convex optimization and/or linear-matrix-inequalities (LMI). And more lately, by Limebeer et al. [179] and Chen and Zhou [81] who considered a two-person nonzero-sum differential game approach with a multi-objective flavor (for the latter). This approach is very transparent and is reminiscent of the minimax approach to $\mathscr{H}$_∞-control by Basar and Bernhard [57]. The state-feedback problem is solved in Limebeer et al. [179], while the output-feedback problem is solved in Chen and Zhou [81]. By-and-large, the outcome of the above endeavors are a parametrization of the solution to the mixed $\mathscr{H}$₂/$\mathscr{H}$_∞-control problem in terms of two cross-coupled nonstandard Riccati equations for the state-feedback problem, and an additional standard Riccati equation for the output-feedback problem.

Similarly, the nonlinear control problem has also been considered more recently by Lin [180]. He extended the results of Limebeer et al. [179], and derived a solution to the state-feedback problem in terms of a pair of cross-coupled Hamilton-Jacobi-Isaac’s equations. In this chapter, we discuss mainly this approach to the problem for both continuous-time and discrete-time nonlinear systems.

11.1  Continuous-Time Mixed $\mathscr{H}$₂/$\mathscr{H}$_∞ Nonlinear Control

In this section, we discuss the mixed $\mathscr{H}$₂/$\mathscr{H}$_∞ nonlinear control problem using state-feedback. The general set-up for this problem is shown in Figure 11.1 with the plant represented by an affine nonlinear system Σ^a, while the static controller is represented by K. The disturbance/noise signal $w=\left(\begin{array}{l}{w}_0\\ w_1\end{array}\right)$, is comprised of two components: (i) a bounded-spectral signal (e.g., a white Gaussian-noise signal) w₀ ∈ $\mathcal{S}$ (the space of bounded-spectral signals), and (ii) a bounded-power signal or $ℒ$₂ signal w₁ ∈ $\mathcal{P}$ (the space of bounded power signals). Thus, the induced norm from the input w₀ to z is the $ℒ$₂-norm of the closed-loop system K ∘ Σ^a, i.e.,

${\Vert K\text{ο}{\sum }^a\Vert }_{{ℒ}_2}\triangleq \underset{0\ne w_0\in \mathcal{S}}{\sup }\frac{{\Vert z\Vert }_p}{{\Vert w_0\Vert }_{\mathcal{S}}},$

(11.1)

FIGURE 11.1
Set-Up for Nonlinear Mixed ${\mathcal{H}}_2$/$\mathcal{H}$_∞ $\mathscr{H}$₂/$\mathscr{H}$_∞-Control

while the induced norm from w₁ to z is the $ℒ$_∞-norm of the closed-loop system K ∘ Σ^a, i.e.,

${\Vert K\text{ο}{\sum }^a\Vert }_{{ℒ}_{\infty }}\triangleq \underset{0\ne w_1\in \mathcal{P}}{\sup }\frac{{\Vert z\Vert }_2}{{\Vert w_1\Vert }_2}$

(11.2)

where

$\begin{array}{l}\mathcal{P}\triangleq \{w(t):w\in {ℒ}_{\infty },R_{ww}(\tau ),S_{ww}(j\omega )existforall\tau \text{and}\text{all}\omega \text{resp}\text{.,}\\ {\Vert w\Vert }_p<\infty \},\\ s\triangleq \{w(t):w\in {ℒ}_{\infty },R_{ww}(\tau ),S_{ww}(j\omega )existforall\tau \text{and}\text{all}\omega \text{resp}\text{.,}\\ {\Vert s_{ww}(j\omega )\Vert }_{\infty }<\infty \},\\ {\Vert z|\Vert }_p^2\triangleq \underset{T\to \infty }{\lim }\frac{1}{2T}{\displaystyle \underset{-T}{\overset{T}{\int }}{\Vert z(t)\Vert }^{2}dt,}s\\ {\Vert w_0\Vert }_s^2={\Vert S_{w_0{w}_0}\left(j\omega \right)\Vert }_{\infty },\end{array}$

and R_ww(τ), S_ww(jω) are the autocorrelation and power-spectral density matrices of w(t) respectively [152]. Notice also that, ${\Vert (.)\Vert }_p\text{and}{\Vert (.)\Vert }_{\mathcal{S}}$ are seminorms. In addition, if the plant is stable, we replace the induced $ℒ$-norms above by their equivalent $\mathscr{H}$-subspace norms. The standard optimal mixed $\mathscr{H}$₂/$\mathscr{H}$_∞ state-feedback control problem is to synthesize a feedback control of the form

$u=\alpha (x),\alpha (0)=0$

(11.3)

such that the above induced-norms (11.1), (11.2) of the closed-loop system are minimized, and the closed-loop system is also locally asymptotically-stable. However, in this chapter, we do not solve the above problem, instead we solve an associated suboptimal $\mathscr{H}$₂/$\mathscr{H}$_∞-control problem which involves a single disturbance w ∈ $ℒ$₂ entering the plant, and where the objective is to minimize the output energy ${\Vert z\Vert }_{{\mathscr{H}}_2}$ of the closed-loop system while rendering ${\Vert K\text{ο}{\sum }^a\Vert }_{\mathscr{H}\infty }\le {\gamma }^{\star }$.

The plant is represented by an affine nonlinear causal state-space system defined on a manifold $\mathcal{X}\subset {\Re }^n$ containing the origin x = 0 :

${\sum }^a:\{\begin{array}{l}\dot{x}=f(x)+g_1(x)w+g_2(x)u;x(t_0)=x_0\\ z=h_1(x)+k_{12}(x)u\\ y=x,\end{array}$

(11.4)

where $x\in \mathcal{X}$ is the state vector, $u\in \mathcal{U}\subseteq {\Re }^p$ is the p-dimensional control input, which belongs to the set of admissible controls $\mathcal{U}\subset {ℒ}_2([t_0,T],{\Re }^p),w\in \mathcal{W}$ is the disturbance signal, which belongs to the set $\mathcal{W}\subset {\Re }^r$ of admissible disturbances (to be defined more precisely later), the output $y\in {\Re }^m$ is the measured output of the system, and $z\in {\Re }^s$ is the output to be controlled. The functions $f:\mathcal{X}\to V^{\infty }(\mathcal{X}),g_1:\mathcal{X}\to {ℳ}^{n\times r}(\mathcal{X}),g_2:\mathcal{X}\to {ℳ}^{n\times p}(\mathcal{X}),h_1:\mathcal{X}\to {\Re }^s\text{and}{k}_{12}:{ℳ}^{n\times p}(\mathcal{X}),$ are real C^∞ functions of x.

Furthermore, we assume without any loss of generality that the system (11.4) has a unique equilibrium-point at x = 0 such that f(0) = 0, h₁(0) = h₂(0) = 0, and for simplicity, we also make the following assumption.

Assumption 11.1.1 The system matrices are such that

$\begin{array}{l}{h}_1^T(x)k_{12}(x)=0\\ k_{12}^T(x)k_{12}(x)=I.\end{array}\}$

(11.5)

The problem can now be formally defined as follows.

Definition 11.1.1 (State-Feedback Mixed $\mathscr{H}$₂/$\mathscr{H}$_∞ Nonlinear Control Problem (SFBMH2HINLCP)).

(A) Finite-Horizon Problem (T < ∞): Find (if possible!) a time-varying static state-feedback control law of the form:

$u={\tilde{\alpha }}_2(x,t),{\tilde{\alpha }}_2(t,0)=0,t\in \Re ,$

such that:

(a) the closed-loop system

${\sum }^{cla}:\{\begin{array}{l}\dot{x}=f(x)+g_{1}w+g_2(x){\tilde{\alpha }}_2(x,t)\\ z=h_1(x)+k_{12}(x){\tilde{\alpha }}_2(x,t)\end{array}$

(11.6)

is stable with w = 0 and has locally finite ${ℒ}_2$-gain from w to z less or equal to γ^* , starting from x₀ = 0, for all t ∈ [0,T ] and all $w\in \mathcal{W}\subseteq {ℒ}_2[0,T];$

(b) the output energy ${\Vert z\Vert }_{{\mathscr{H}}_2}$ of the system is minimized.

(B) Infinite-Horizon Problem (T → ∞) : In addition to the items (a) and (b) above, it is also required that

(c) the closed-loop system Σ^cla defined above with w ≡ 0 is locally asymptotically-stable about the equilibrium-point x = 0.

Such a problem can be formulated as a two-player nonzero-sum differential game (Chapter 2) with two cost functionals:

$\underset{u\in \mathcal{U},w\in \mathcal{W}}{\min }{J}_1(u,w)=\frac{1}{2}{\displaystyle \underset{t_0}{\overset{T}{\int }}({\gamma }^2{\Vert w(\tau )\Vert }^2-{\Vert z(\tau )\Vert }^2)d\tau }$

(11.7)

$\underset{u\in \mathcal{U},w\in \mathcal{W}}{\min }{J}_2(u,w)=\frac{1}{2}{\displaystyle \underset{t_0}{\overset{T}{\int }}{\Vert z(\tau )\Vert }^{2}d\tau }$

(11.8)

for the finite-horizon problem, with T ≥ t₀. Here, the first functional is associated with the $\mathscr{H}$_∞-constraint criterion, while the second functional is related to the output energy of the system or $\mathscr{H}$₂-criterion. It can easily be seen that, by making J₁ ≥ 0, the $\mathscr{H}$_∞-constraint ${\Vert K\text{ο}{\sum }^a\Vert }_{\mathscr{H}\infty }={\Vert {\sum }^{cla}\Vert }_{\mathscr{H}\infty }\le \gamma$ is satisfied. Subsequently, minimizing J₂ will achieve the $\mathscr{H}$₂/$\mathscr{H}$_∞ design objective. A Nash-equilibrium solution to the above game is said to exist if we can find a pair of strategies (u^*, w^⋆) such that

$J_1(u^{\star },w^{\star })\le J_1(u^{\star },w)\forall w\in \mathcal{W},$

(11.9)

$J_2(u^{\star },w^{\star })\le J_2(u,w^{\star })\forall u\in \mathcal{U}.$

(11.10)

Furthermore, by minimizing the first objective with respect to w and substituting in the second objective which is then minimized with respect to u, the above pair of Nash-equilibrium strategies could be found. A sufficient condition for the solvability of the above differential game is provided from Theorem 2.3.1, Chapter 2, by the following pair of cross-coupled HJIEs for the finite-horizon state-feedback problem:

$\begin{array}{l}-Y_t(x,t)=\underset{w\in \mathcal{W}}{\inf }\left\{Y_x(x,t)f(x,u^{\star }(x),w(x))+{\gamma }^2{\Vert w(x)\Vert }^2-{\Vert z^{\star }(x)\Vert }^2\right\},Y(x,T)=0,\\ -V_t(x,t)=\underset{u\in \mathcal{U}}{\min }\left\{V_x(x,t)f(x,u(x),w^{\star }(x))+{\Vert z^{\star }(x)\Vert }^2\right\}+0,V(x,T)=0,\end{array}$

for some negative-definite function $Y:\mathcal{X}\to \Re$ and positive-definite function $V:\mathcal{X}\to \Re$, where $z^{\star }(x)=h_1(x)+k_{12}(x)$u^*(x).

In view of the above result, the following theorem gives sufficient conditions for the solvability of the finite-horizon problem.

Theorem 11.1.1 Consider the nonlinear system Σ^a defined by (11.4) and the finite-horizon SFBMH2HINLCP with cost functionals (11.7), (11.8). Suppose there exists a pair of negative and positive-definite C¹-functions (with respect to both arguments) Y, V : N ×[0,T ] →$\mathrm{\Re }$ locally defined in a neighborhood N of the origin x = 0, such that Y (0,t) = 0 and V (0,t) = 0, and satisfying the coupled HJIEs:

$\begin{array}{l}-Y_t(x,t)=Y_x(x,t)f(x)-\frac{1}{2}{V}_x(x,t)g_2(x)g_2^T(x)V_x^T(x,t)-\\ \frac{1}{2{\gamma }^2}{Y}_x(x,t)g_1(x)g_1^T(x)Y_x^T(x,t)-Y_x(x,t)g_2(x)g_2^T(x)V_x^T(x,t)-\\ \frac{1}{2}{h}_1^T(x)h_1(x),Y(x,T)=0\end{array}$

(11.11)

$\begin{array}{l}-V_t(x,t)=V_x(x,t)f(x)-\frac{1}{2}{V}_x(x,t)g_2(x)g_2^T(x)V_x^T(x,t)-\\ \frac{1}{{\gamma }^2}{V}_x(x,t)g_1(x)g_1^T(x)Y_x^T(x,t)+\frac{1}{2}{h}_1^T(x)h_1(x),V(x,T)=0.\end{array}$

(11.12)

Then the state-feedback controls

$u^{\star }(x,t)=-g_2^T(x)V_x^T(x,t)$

(11.13)

$w^{\star }(x,t)=-\frac{1}{{\gamma }^2}{g}_1^T(x)Y_x^T(x,t)$

(11.14)

solve the finite-horizon SFBMH2HINLCP for the system. Moreover, the optimal costs are given by

$J_1^{\star }(u^{\star },w^{\star })=Y(t_0,x_0)$

(11.15)

$J_2^{\star }(u^{\star },w^{\star })=V(t_0,x_0).$

(11.16)

Proof: Assume there exists locally solutions Y < 0, V > 0 to the HJIEs (11.11), (11.12) in $N\subset \mathcal{X}$. We prove item (a) of Definition 11.1.1 first. Rearranging the HJIE (11.11) and completing the squares, we have

$\begin{array}{l}{Y}_t+Y_x(f(x)+g_1(x)w-g_2(x)g^T(x)V_x^T(x))=\frac{1}{2}{\Vert h_1(x)\Vert }^2+\frac{{\gamma }^2}{2}{\Vert w-w^{\star }\Vert }^2+\\ \frac{1}{2}{\Vert u^{\star }\Vert }^2-\frac{1}{2}{\gamma }^2{\Vert w\Vert }^2\\ \iff \dot{Y}(x,t)=\frac{1}{2}{\Vert z|\Vert }^2-\frac{1}{2}{\gamma }^2{\Vert w\Vert }^2+\frac{{\gamma }^2}{2}{\Vert w-w^{\star }\Vert }^2\\ \iff \dot{\tilde{Y}}(x,t)\le \frac{1}{2}{\gamma }^2{\Vert w\Vert }^2-\frac{1}{2}{\Vert z\Vert }^2\end{array}$

for some function $\tilde{Y}=-Y>0$. Integrating now the above expression from t₀ and x(t₀) to t = T and x(T ), we get the dissipation-inequality

$\tilde{Y}(x(T),T)-\tilde{Y}(x(t_0,t_0)\le {\displaystyle \underset{t_0}{\overset{T}{\int }}\frac{1}{2}({\gamma }^2{\Vert w\Vert }^2-{\Vert z\Vert }^2)dt.}$

Therefore, the system has locally ${ℒ}_2-\text{gain}\le \gamma$ from w to z. Furthermore, the closed-loop system with u = u^* and w = 0 is given by

$\dot{x}=f(x)-g_2(x)g_2^T(x)V_x^T(x).$

Differentiating $\tilde{Y}$ from above along a trajectory of this system, we have

$\begin{array}{l}\dot{\tilde{Y}}={\tilde{Y}}_t(x,t)+{\tilde{Y}}_t(x,t)(f(x)-g_2(x)g_2^T(x)V_x^T)\forall x\in N\\ \le -\frac{1}{2}{\Vert z\Vert }^2\le 0.\end{array}$

Hence, the closed-loop system is Lyapunov-stable.

Next we prove item (b). Consider the cost functional J₁(u,w) first. For any T ≥ t₀, the following holds

$\begin{array}{l}{J}_1(u,w)+Y(x(T),T)-Y(x(t_0),t_0)=\text{}\text{}{\displaystyle \underset{t_0}{\overset{T}{\int }}\{\frac{1}{2}({\gamma }^2{\Vert w\Vert }^2-{\Vert z\Vert }^2)+}\dot{Y}(x,t)\}dt\\ ={\displaystyle \underset{t_0}{\overset{T}{\int }}\left\{\frac{1}{2}({\gamma }^2{\Vert w\Vert }^2-{\Vert z|\Vert }^2)+Y_t+Y_x(f(x)+g_1(x)w+g_2(x)u\right\}+}dt\\ ={\displaystyle \underset{t_0}{\overset{T}{\int }}\{}{Y}_t+Y_{x}f(x)Y_x{g}_2(x)u+\frac{{\gamma }^2}{2}{\Vert w+\frac{1}{{\gamma }^2}{g}_1^T(x)Y_x^T\Vert }^2-\frac{{\gamma }^2}{2}{\Vert \frac{1}{{\gamma }^2}{g}_1^T(x)Y_x^T\Vert }^2\\ -\frac{1}{2}{\Vert u\Vert }^2-\frac{1}{2}{\Vert h_1(x)\Vert }^2\}dt.\end{array}$

Using the HJIE (11.11), we have

$\begin{array}{l}{J}_1(u,w)+Y(x(T),T)-Y(x(t_0),t_0)={\displaystyle \underset{t_0}{\overset{T}{\int }}\{\frac{{\gamma }^2}{2}{\Vert w+\frac{1}{{\gamma }^2}{g}_1^T(x)Y_x^T\Vert }^2}-\\ \frac{1}{2}{\Vert u+g_2^T(x)V_x^T\Vert }^2+\\ {\Vert g_2^T(x)V_x^T\Vert }^2+V_x(x)g_2(x)u+\\ Y_x{g}_2(x)(u-g_2^T(x)V_x^T(x))\}dt,\end{array}$

and substituting u = u^*, we have

$J_1(u^{\star },w)+Y(x(T),T)-Y(x(t_0),t_0)={\displaystyle \underset{t_0}{\overset{T}{\int }}\frac{{\gamma }^2}{2}\Vert w-w^{\star }\Vert dt\ge 0.}$

Therefore

$J_1(u^{\star },w^{\star })\le J_1(u^{\star },w)$

with

$J_1(u^{\star },w^{\star })=Y(x(t_0),t_0)$

since Y (x(T),T ) = 0.

Similarly, considering the cost functional J₂(u, w), the following holds for any T > 0

$\begin{array}{l}{J}_2(u,w)+V(x(T),T)-V(x(t_0),t_0)={\displaystyle \underset{t_0}{\overset{T}{\int }}\left\{\frac{1}{2}{\Vert z\Vert }^2+\dot{V}(x,t)\right\}}\\ ={\displaystyle \underset{t_0}{\overset{T}{\int }}\{\frac{1}{2}{\Vert z\Vert }^2+V_t(x,t)+V_x(f(x)+}\\ g_1(x)w+g_2(x)u)\}dt\\ ={\displaystyle \underset{t_0}{\overset{T}{\int }}\{V_t+V_{x}f(x)+V_x{g}_1}\\ \frac{1}{2}{\Vert u+g_2^T{V}_x^T\Vert }^2-{\Vert g_2^T(x)V_x^T\Vert }^2+\frac{1}{2}{\Vert h_1^T(x)\Vert }^2\}dt.\end{array}$

Using the HJIE (11.12) in the above, we get

$J_2(u,w)+V(x(T),T)-V(x(t_0),t_0)={\displaystyle \underset{t_0}{\overset{T}{\int }}\left\{\frac{1}{2}{\Vert u+g_2^T{V}_x^T\Vert }^2+V_x{g}_1(x)(w-\frac{1}{{\gamma }^2}{g}_1^T{Y}_x^T)\right\}}dt.$

Substituting now w = w^⋆ we get

$J_2(u,w^{\star })+V(x(T),T)-V(x(t_0),t_0)={\displaystyle \underset{t_0}{\overset{T}{\int }}\frac{1}{2}{\Vert u+g_2^T{V}_x^T\Vert }^2}dt\ge 0,$

and therefore

$J_2(u^{\star },w^{\star })\le J_2(u,w^{\star })$

with

$J_2(u^{\star },w^{\star })=V(x(t_0),t_0).\square$

We can specialize the results of the above theorem to the linear system

${\sum }^l:\{\begin{array}{l}\dot{x}=Ax+B_{1}w+B_{2}u\\ z=C_{1}x+D_{12}w\\ y=x\end{array}$

(11.17)

under the following assumption.

Assumption 11.1.2

$C_1^T{D}_{12}=0,D_{12}^T{D}_{12}=I.$

Then we have the following corollary.

Corollary 11.1.1 Consider the linear system Σ^l under the Assumption 11.1.2. Suppose there exist P₁(t) ≤ 0 and P₂(t) ≥ 0 solutions of the cross-coupled Riccati ordinary-differential-equations (ODEs):

$\begin{array}{l}-{\dot{P}}_1(t)=A^T{P}_1+P_1(t)A-[P_1(t)P_2(t)]\left[\begin{array}{cc}{\gamma }^{-2}{B}_1{B}_1^T& B_2{B}_2^T\\ B_2{B}_2^T& B_2{B}_2^T\end{array}\right]\left[\begin{array}{l}{P}_1(t)\\ P_2(t)\end{array}\right]\_C_1^T{C}_1,\\ P_1(T)=0\\ -{\dot{P}}_2(t)=A^T{P}_2+P_2(t)A-[P_1(t)P_2(t)]\left[\begin{array}{cc}0& {\gamma }^{-2}{B}_1{B}_1^T\\ {\gamma }^{-2}{B}_1{B}_1^T& B_2{B}_2^T\end{array}\right]\left[\begin{array}{l}{P}_1(t)\\ P_2(t)\end{array}\right]\_C_1^T{C}_1,\\ P_2(T)=0\end{array}$

on [0,T]. Then, the Nash-equilibrium strategies uniquely specified by

$\begin{array}{l}{u}^{\star }=-B_2^T{P}_2(t)x(t)\\ w^{\star }\text{}\text{}\text{}\text{}=-\frac{1}{{\gamma }^2}{B}_1^T{P}_1(t)x(t)\text{}\end{array}$

solve the finite-horizon SFBMH2HICP for the system. Moreover, the optimal costs for the game associated with the system are given by

$\begin{array}{l}{J}_1(u^{\star },w^{\star })=\frac{1}{2}{x}^T(t_0)P_1(t_0)x(t_0),\\ J_2(u^{\star },w^{\star })=\frac{1}{2}{x}^T(t_0)P_2(t_0)x(t_0).\end{array}$

Proof: Take

$\begin{array}{l}Y(x(t),t)=\frac{1}{2}{x}^T(t)P_1(t)x(t),P_1(t)\le 0\\ V(x(t),t)=\frac{1}{2}{x}^T(t)P_2(t)x(t),P_2(t)\ge 0\end{array}$

and apply the results of the Theorem. □

Remark 11.1.1 In the above corollary, we considered negative and positive-(semi)definite solutions of the Riccati (ODEs), while in Theorem 11.1.1 we considered strict definite solutions of the HJIEs. However, it is generally sufficient to consider semidefinite solutions of the HJIEs.

11.1.1  The Infinite-Horizon Problem

In this subsection, we consider the infinite-horizon SFBMH2HINLCP for the affine nonlinear system Σ^a. In this case, we let T → ∞, and seek time-invariant functions and feedback gains that solve the HJIEs. Because of this, it is necessary to require that the closed-loop system is locally asymptotically-stable as stated in item (c) of the definition. However, to achieve this, some additional assumptions on the system might be necessary. The following theorem gives sufficient conditions for the solvability of this problem. We recall the definition of detectability first.

Definition 11.1.2 The pair {f,h} is said to be locally zero-state detectable if there exists a neighborhood $\mathcal{O}$ of x = 0 such that, if x(t) is a trajectory of $\dot{x}$(t) = f(x) satisfying $x(t_0)\in \mathcal{O}$, then h(x(t)) is defined for all t ≥ t₀, and h(x(t)) ≡ 0, for all t ≥ t_s, implies ${\lim }_{t\to \infty }x(t)=0$. Moreover,{f,h} is detectable if $\mathcal{O}=\mathcal{X}$.

Theorem 11.1.2 Consider the nonlinear system Σ^a defined by (11.4) and the infinitehorizon SFBMH2HINLCP with cost functions (11.7), (11.8). Suppose

(H1) the pair {f,h₁} is zero-state detectable;

(H2) there exists a pair of negative and positive-definite C¹-functions $\tilde{Y},\tilde{V}:\tilde{N}\to \Re$ locally defined in a neighborhood $\tilde{N}$of the origin x = 0, and satisfying the coupled HJIEs:

$\begin{array}{l}{\tilde{Y}}_x(x)f(x)-\frac{1}{2}{\tilde{V}}_x(x)g_2(x)g_2^T(x){\tilde{V}}_x^T(x)-\frac{1}{2{\gamma }^2}{\tilde{Y}}_x(x)g_1(x)g_1^T(x){\tilde{Y}}_x^T(x)-\\ {\tilde{Y}}_x(x)g_2(x)g_2^T(x){\tilde{V}}_x^T(x)-\frac{1}{2}{h}_1^T(x)h_1(x)=0,\tilde{Y}(0)=0\end{array}$

(11.18)

$\begin{array}{l}{\tilde{V}}_x(x)f(x)-\frac{1}{2}{\tilde{V}}_x(x)g_2(x)g_2^T(x){\tilde{V}}_x^T(x)-\frac{1}{{\gamma }^2}{\tilde{V}}_x(x)g_1(x)g_1^T(x){\tilde{Y}}_x^T(x)+\\ \frac{1}{2}{h}_1^T(x)h_1(x)=0,\tilde{V}(0)=0.\end{array}$

(11.19)

Then the state-feedback controls

$u^{\star }(x)=-g_2^T(x){\tilde{V}}_x^T(x)$

(11.20)

$w^{\star }(x)=-\frac{1}{{\gamma }^2}{g}_1^T(x){\tilde{Y}}_x^T(x)$

(11.21)

solve the infinite-horizon SFBMH2HINLCP for the system. Moreover, the optimal costs are given by

$J_1^{\star }(u^{\star },w^{\star })=\tilde{Y}(x_0)$

(11.22)

$J_2^{\star }(u^{\star },w^{\star })=\tilde{V}(x_0).$

(11.23)

Proof: We only prove item (c) in the definition, since the proofs of items (a) and (b) are similar to the finite-horizon case. Using similar manipulations as in the proof of item (a) of Theorem 11.1.1, it can be shown that with w ≡0,

$\dot{\stackrel{⌣}{Y}}=-\frac{1}{2}{\Vert z\Vert }^2,$

for some function $\stackrel{⌣}{Y}=-\tilde{Y}$ > 0. Therefore the closed-loop system is Lyapunov-stable. Further, the condition $\dot{\stackrel{⌣}{Y}}\equiv 0\forall t\ge t_s$, for some t_s ≥ t₀, implies that u^* ≡ 0, h₁(x) ≡ 0. By hypothesis (H1), this implies lim_t→∞ x(t) = 0, and we can conclude asymptotic-stability by LaSalle’s invariance-principle. □

The above theorem can again be specialized to the linear system Σ^l in the following corollary.

Corollary 11.1.2 Consider the linear system Σ^l under the Assumption 11.1.2. Suppose (C₁, A) is detectable and there exists symmetric solutions ${\overline{P}}_1\le 0and{\overline{P}}_2\ge 0$ of the cross-coupled AREs:

$A^T{\overline{P}}_1+{\overline{P}}_{1}A-[{\overline{P}}_1{\overline{P}}_2]\left[\begin{array}{cc}{\gamma }^{-2}{B}_1{B}_1^T& B_2{B}_2^T\\ B_2{B}_2^T& B_2{B}_2^T\end{array}\right][\begin{array}{l}{\overline{P}}_1\\ {\overline{P}}_2\end{array}+]\_C_1^T{C}_1=0,$

$A^T{\overline{P}}_2+{\overline{P}}_{2}A-[{\overline{P}}_1{\overline{P}}_2]\left[\begin{array}{cc}0& {\gamma }^{-2}{B}_1{B}_1^T\\ {\gamma }^{-2}{B}_1{B}_1^T& B_2{B}_2^T\end{array}\right]\left[\begin{array}{l}{\overline{P}}_1\\ {\overline{P}}_2\end{array}\right]+C_1^T{C}_1=0$

Then, the Nash-equilibrium strategies uniquely specified by

$u^{\star }=-B_2^T{\overline{P}}_2(t)x(t)$

(11.24)

$w^{\star }=-\frac{1}{{\gamma }^2}{B}_1^T{\overline{P}}_{1}x(t)$

(11.25)

solve the infinite-horizon SFBMH2HINLCP for the system. Moreover, the optimal costs for the game associated with the system are given by

$J_1(u^{\star },w^{\star })=\frac{1}{2}{x}^T(t_0){\overline{P}}_{1}x(t_0),$

(11.26)

$J_2(u^{\star },w^{\star })=\frac{1}{2}{x}^T(t_0){\overline{P}}_{2}x(t_0).$

(11.27)

Proof: Take

$\begin{array}{l}Y(x,t)=\frac{1}{2}{x}^T{\overline{P}}_{1}x,{\overline{P}}_1\le 0,\\ V(x,t)=\frac{1}{2}{x}^T{\overline{P}}_{2}x,{\overline{P}}_2\le 0.\end{array}$

and apply the result of the theorem. □

Remark 11.1.2 Sufficient conditions for the existence of the asymptotic solutions to the coupled algebraic-Riccati equations are discussed in Reference [222].

The following proposition can also be proven.

Proposition 11.1.1 Consider the nonlinear system (11.4) and the infinite-horizon SFBMH2HINLCP for this system. Suppose the following hold:

(a1) {f + g₁w^⋆, h₁} is locally zero-state detectable;

(a2) there exists a pair of C¹ negative and positive-definite functions $\tilde{Y},\tilde{V}$ : $\tilde{N}\to \Re$ respectively, locally defined in a neighborhood $N\subset \mathcal{X}$ of the origin x = 0 satisfying the pair of coupled HJIEs (11.18), (11.19).

Then $f-g_2{g}_2^T{\tilde{V}}_x^T-\frac{1}{{\gamma }^2}{g}_1{g}_1^T{\tilde{Y}}_x^T$ is locally asymptotically-stable.

Proof: Rewrite the HJIE (11.19) as

$\begin{array}{l}(f(x)-g_2(x)g_2^T(x){\tilde{V}}_x^T(x)-\frac{1}{2}{g}_1^T(x)g_1^T(x){\tilde{Y}}_x(x))=-\frac{1}{2}{\Vert g_2^T(x){\tilde{V}}_x^T(x)\Vert }^2-\frac{1}{2}{\Vert h_1(x)\Vert }^2\\ \iff \dot{\tilde{V}}=-\frac{1}{2}{\Vert g_2^T(x){\tilde{V}}_x^T(x)\Vert }^2-\frac{1}{2}{\Vert h_1(x)\Vert }^2\\ \le 0.\end{array}$

Again the condition $\dot{\tilde{V}}\equiv 0\forall t\ge t_s$, implies that $u^{\star }\equiv 0,h_1(x)\equiv 0\forall t\ge t_s$. By Assumption (a1), this implies lim_t→∞ x(t) = 0, and by the LaSalle’s invariance-principle, we conclude asymptotic-stability of the vector-field $f-g_2{g}_2^T{\tilde{V}}_x^T-\frac{1}{{\gamma }^2}{g}_1{g}_1^T{\tilde{Y}}_x^T$.□

Remark 11.1.3 The proof of the equivalent result for the linear system Σ^l can be pursued along the same lines. Moreover, many interesting corollaries could also be derived (see also Reference [179]).

11.1.2  Extension to a General Class of Nonlinear Systems

In this subsection, we consider the SFBMH2HINLCP for a general class of nonlinear systems which are not necessarily affine, and extend the approach developed above to this class of systems. We consider the class of nonlinear systems described by

$\sum :\{\begin{array}{l}\dot{x}=F(x,w,u),x(t_0)=x_0\\ z=Z(x,u)\\ y=x\end{array}$

(11.28)

where all the variables have their previous meanings and dimensions, while $F:\mathcal{X}\times \mathcal{W}\times \mathcal{U}\to {\Re }^n,Z:\mathcal{X}\times \mathcal{U}\to {\Re }^s$ are smooth functions of their arguments. It is also assumed that F(0, 0, 0) = 0 and Z(0, 0) = 0 and the system has a unique equilibrium-point at x = 0. The infinite-horizon SFBMH2HINLCP for the above system can similarly be formulated as a dynamic two-person nonzero-sum differential game with the same cost functionals (11.7), (11.8). In addition, we also assume the following for simplicity.

Assumption 11.1.3 The system matrices satisfy the following conditions:

(i) $\det \left[{\left(\frac{\partial Z}{\partial u}(0,0)\right)}^T\left(\frac{\partial Z}{\partial u}(0,0)\right)\right]\ne 0;$

(ii) $Z(x,u)=0\Rightarrow u=0$

Define now the Hamiltonian functions $K_i:T^{\star }\mathcal{X}\times \mathcal{W}\times \mathcal{U}\to \Re ,i=1,2$ corresponding to the cost functionals (11.7), (11.8) respectively:

$\begin{array}{l}{K}_1(x,w,u,{\overline{Y}}_x^T)={\overline{Y}}_x(x)F(x,w,u)+\frac{1}{2}{\gamma }^2{\Vert w\Vert }^2-{\Vert z(x,u)\Vert }^2\\ K_2(x,w,u,{\overline{V}}_x^T)={\overline{V}}_x(x)F(x,w,u)+\frac{1}{2}{\Vert z(x,u)\Vert }^2\end{array}$

for some smooth functions $\overline{Y},\overline{V}:\mathcal{X}\to \Re ,\text{and}\text{where}{p}_1={\overline{Y}}_x^T,p_2={\overline{V}}_x^T$ are the adjoint variables. Then, it can be shown that

$\begin{array}{l}{{\partial }^{2}K|}_{w=0,u=0}(0)\triangleq {\left[\begin{array}{cc}\frac{{\partial }^2{K}_2}{\partial u^2}& \frac{{\partial }^2{K}_2}{\partial w\partial u}\\ \frac{{\partial }^2{K}_1}{\partial u\partial w}& \frac{{\partial }^2{K}_1}{\partial w^2}\end{array}\right]|}_{w=0,u=0}(0)\\ =\left[\begin{array}{cc}{\left(\frac{\partial Z}{\partial u}(0,0)\right)}^T\left(\frac{\partial Z}{\partial u}(0,0)\right)& 0\\ 0& {\gamma }^{2}I\end{array}\right]\end{array}$

is nonsingular by Assumption 11.1.3. Therefore, by the Implicit-function Theorem, there exists an open neighborhood $\overline{X}$ of x = 0 such that the equations

$\frac{\partial K_1}{\partial w}(x,{\overline{w}}^{\star }(x),{\overline{u}}^{\star }(x))=0,$

$\frac{\partial K_2}{\partial u}(x,{\overline{w}}^{\star }(x),{\overline{u}}^{\star }(x))=0$

have unique solutions $({\overline{u}}^{\star }(x),{\overline{w}}^{\star }(x)),\text{with}{\overline{u}}^{\star }(0)=0,{\overline{w}}^{\star }(0)=0$. Moreover, the pair $({\overline{u}}^{\star },{\overline{w}}^{\star })$ constitutes a Nash-equilibrium solution to the dynamic game (11.28), (11.7), (11.8). The following theorem summarizes the solution to the infinite-horizon problem for the general class of nonlinear systems (11.28).

Theorem 11.1.3 Consider the nonlinear system (11.28) and the SFBMH2HINLCP for it. Suppose Assumption 11.1.3 holds, and also the following:

(A1) the pair {F (x, 0, 0),Z(x,0)} is zero-state detectable;

(A2) there exists a pair of C¹ locally negative and positive-definite functions $\overline{Y},\overline{V}:\overline{N}\to \Re$

respectively, defined in a neighborhood $\overline{N}$ of x = 0, vanishing at x = 0 and satisfying the pair of coupled HJIEs:

${\overline{Y}}_x(x)F(x,{\overline{w}}^{\star }(x),{\overline{u}}^{\star }(x))+\frac{1}{2}{\gamma }^2{\Vert {\overline{w}}^{\star }(x)\Vert }^2-\frac{1}{2}{\Vert Z(x,{\overline{u}}^{\star }(x))\Vert }^2=0,\overline{Y}(0)=0,$

${\overline{V}}_x(x)F(x,{\overline{w}}^{\star }(x),{\overline{u}}^{\star }(x))+\frac{1}{2}{\Vert Z(x,{\overline{u}}^{\star }(x))\Vert }^2=0,\overline{V}(0)=0;$

(A3) the pair $\{F(x,{\overline{w}}^{\star }(x),0),Z(x,0)\}$ is zero-state detectable.

Then, the state-feedback controls $({\overline{u}}^{\star }(x),{\overline{w}}^{\star }(x))$ solve the dynamic game problem and the SFBMH2HINLCP for the system (11.28). Moreover, the optimal costs of the policies are given by

$\begin{array}{l}{\overline{J}}_1^{\star }({\overline{u}}^{\star },{\overline{w}}^{\star })=\overline{Y}(x_0),\\ {\overline{J}}_2^{\star }({\overline{u}}^{\star },{\overline{w}}^{\star })=\overline{V}(x_0).\end{array}$

Proof: The proof can be pursued along the same lines as the previous results. □