11.2  Discrete-Time Mixed ${\mathcal{H}}_2$/${\mathcal{H}}_{\mathrm{\infty }}$ Nonlinear Control

In this section, we discuss the state-feedback mixed ${\mathcal{H}}_2$/${\mathcal{H}}_{\mathrm{\infty }}$-control problem for discrete-time nonlinear systems. We begin with an affine discrete-time state-space model defined on $\mathcal{X}\subset {\Re }^n$ in coordinates (x₁,…,x_n) :

${\sum }^{da}:\{\begin{array}{l}{\dot{x}}_{k+1}=f(x_k)+g_1(x_k)w_k+g_2(x_k)u_k;x(k_0)=x^0\\ z_k=h_1(x_k)+k_{12}(x_k)u_k\\ y_k=x_k,\end{array}$

(11.29)

where all the variables and system matrices have their previous meanings and dimensions respectively. We assume similarly that the system has a unique equilibrium at x = 0, and is such that f(0) = 0 and h₁0) = 0. For simplicity, we similarly also assume the following hold for the system matrices.

Assumption 11.2.1 The system matrices are such that

$\begin{array}{l}{h}_1^T(x)k_{12}(x)=0,\\ k_{12}^T(x)k_{12}(x)=I.\end{array}\}$

(11.30)

Again, as in the continuous-time case, the standard problem is to design a static state-feedback controller, K^da, such that the ℌ₂-norm of the closed-loop system which is defined as

${\Vert K^{da}\text{ο}{\sum }^{da}\Vert }_{{\ell }_2}\triangleq \underset{0\ne w_0\in \mathcal{S}}{\sup }\frac{\Vert z\Vert \mathcal{P}}{\Vert w_0\Vert {\mathcal{S}}^{\prime }},$

and the ${\mathcal{H}}_{\infty }$-norm of the system defined by

${\Vert K^{da}\text{ο}{\sum }^{da}\Vert }_{{\ell }_{\infty }}\triangleq \underset{0\ne w_1\in {\mathcal{P}}^{\prime }}{\sup }\frac{{\Vert z\Vert }_2}{{\Vert w_1\Vert }_2},$

are minimized over a time horizon [k₀, K] ⊂ Z, where

$\begin{array}{l}{\mathcal{P}}^{\prime }\triangleq \{w:\text{}w\in {\ell }_{\infty },R_{ww}(k),S_{ww}(j\omega )\text{exist}\text{for}\text{all}k\text{and}\text{all }\omega \text{resp}\text{.,}\\ \Vert w\Vert {\mathcal{P}}^{\prime }<\infty \}\end{array}$

$\begin{array}{l}{\mathcal{S}}^{\prime }\triangleq \{w:\text{}w\in {\ell }_{\infty },R_{ww}(k),S_{ww}(j\omega )\text{exist}\text{for}\text{all}k\text{and}\text{all }\omega \text{resp}\text{.,}\\ {\Vert S_{ww}\text{(}j\omega )\Vert }_{\infty }<\infty \}\end{array}$

$\begin{array}{l}{\Vert z\Vert }_{{\mathcal{P}}^{\prime }}^2\triangleq \underset{k\to \infty }{\lim }\frac{1}{2K}{\displaystyle \sum _{k=-K}^K{\Vert z_k\Vert }^2}\\ {\Vert w_0\Vert }_{{\mathcal{S}}^{\prime }}^2={\Vert S_{w_0{w}_0}(jw)\Vert }_{\infty }\end{array}$

and $R_{ww},S_{ww}(j\omega ))$ are the autocorrelation and power spectral density matrices of w [152]. Notice also that ${\Vert (.)\Vert }_{{\mathcal{P}}^{\prime }}\text{and}{\Vert (.)\Vert }_{{\mathcal{S}}^{\prime }}$ are seminorms. The spaces ${\mathcal{P}}^{\prime }\text{and}{\mathcal{S}}^{\prime }$ are the discrete-time spaces of bounded-power and bounded-spectral signals respectively.

However, we do not solve the above standard problem in this section. Instead, we solve an associated suboptimal problem in which w ∈ ℓ₂[k₀, ∞) is a single disturbance, and the objective is to minimize the output energy ${\Vert z\Vert }_{{\ell }_2}$ subject to the constraint that ${\Vert {\sum }^{da}\Vert }_{{\ell }_{\infty }}\le {\gamma }^{\star }$ for some number ${\gamma }^{\star }>0$. The problem is more formally defined as follows.

Definition 11.2.1 (Discrete-Time State-Feedback Mixed H₂ / ${\mathcal{H}}_{\mathrm{\infty }}$ Nonlinear Control Problem (DSFBMH2HINLCP)).

(A) Finite-Horizon Problem (K < ∞): Find (if possible!) a time-varying static state-feedback control law of the form:

$u={\tilde{\alpha }}_{2d}(k,x_k),{\tilde{\alpha }}_{2d}(k,0)=0,k\in Z$

such that:

(a) the closed-loop system

${\sum }^{clda}:\{\begin{array}{l}{x}_{k+1}=f(x_k)+g_1(x_k)w_k+g_2(x_k){\tilde{\alpha }}_{2d}(x_k)\\ z_k=h_1(x_k)+k_{21}(x_k){\tilde{\alpha }}_{2d}(x_k)\end{array}$

(11.31)

is stable with w = 0 and has locally finite ℓ₂-gain from w to z less or equal to γ^⋆, starting from x⁰ = 0, for all k ∈ [k₀, K] and for a given number γ^⋆ > 0.

(b) the output energy ${\Vert z\Vert }_{{\ell }_2}$ of the system is minimized for all disturbances $w\in \mathcal{W}\subset {\ell }_2[k_0,\infty )$.

(B) Infinite-Horizon Problem (K → ∞ ): In addition to the items (a) and (b) above, it is also required that

(c) the closed-loop system Σ^clda defined above with w ≡ 0 is locally asymptotically-stable about the equilibrium-point x = 0.

The problem is similarly formulated as a two-player nonzero-sum differential game with two cost functionals:

$\underset{u\in \mathcal{U},w\in \mathcal{W}}{\min }{J}_{1d}(u,w)=\frac{1}{2}{\displaystyle \sum _{k=k_0}^K({\gamma }^2{\Vert w_k\Vert }^2-{\Vert z_k\Vert }^2)}$

(11.32)

$\underset{u\in \mathcal{U},w\in \mathcal{W}}{\min }{J}_{2d}(u,w)=\frac{1}{2}{\displaystyle \sum _{k=k_0}^K{\Vert z_k\Vert }^2.}$

(11.33)

Again, sufficient conditions for the solvability of the above dynamic game (11.32), (11.33), (11.29), and the existence of Nash-equilibrium strategies are given by the following pair of discrete-time Hamilton-Jacobi-Isaac’s difference equations (DHJIEs):

$\begin{array}{l}W(x,k)=\underset{w}{\inf }\left\{W(f(x)+g_1(x)w+g_2(x)u^{\star },k+1)+\frac{1}{2}({\gamma }^2{\Vert w_k\Vert }^2-{\Vert z_k^{\star }\Vert }^2)\right\},\\ W(x,K+1)=0,\end{array}$

(11.34)

$\begin{array}{l}U(x,k)=\underset{u}{\inf }\left\{U(f(x)+g_1(x)w^{\star }+g_2(x)u,k+1)+\frac{1}{2}{\Vert z_k^{\star }\Vert }^2\right\},\\ U(x,K+1)=0,\end{array}$

(11.35)

for some smooth negative and positive-definite functions $W,U:\mathcal{X}\times Z\to \Re$ respectively, and where $z_k^{\star }=h_1(x)+k_{12}(x)u^{\star }(x)$.

To solve the problem, we define the Hamiltonian functions $H_i:\mathcal{X}\times \mathcal{W}\times \mathcal{U}\to \Re \to \Re$, i = 1,2 corresponding to the cost functionals (11.32), (11.32) respectively and the system equations (11.29):

$\begin{array}{l}{H}_1(x,w,u,W)=W(f(x)+g_1(x)w+g_2(x)u,k+1)-W(x)+\\ \frac{1}{2}({\gamma }^2{\Vert w\Vert }^2-\Vert z_k\Vert ),\end{array}$

(11.36)

$\begin{array}{l}{H}_2(x,w,u,U)=U(f(x)+g_1(x)w+g_2(x)u,k+1)-U(x)+\\ \frac{1}{2}{\Vert z_k\Vert }^2.\end{array}$

(11.37)

Similarly, as in Chapter 8, let

$\begin{array}{l}{\partial }^{2}H(x)\triangleq \left[\begin{array}{cc}\frac{{\partial }^2{H}_2}{\partial u^2}& \frac{{\partial }^2{H}_2}{\partial w\partial u}\\ \frac{{\partial }^2{H}_1}{\partial u\partial w}& \frac{{\partial }^2{H}_1}{\partial w^2}\end{array}\right](x)\triangleq \left[\begin{array}{cc}{r}_{11}(x)& r_{12}(x)\\ r_{21}(x)& r_{22}(x)\end{array}\right]\\ F^{\star }(x)\triangleq f(x)+g_1(x)w^{\star }(x)+g_2(x)u^{\star }(x),\end{array}$

(11.38)

and therefore

$\begin{array}{l}{r}_{11}(x)=g_2^T(x){\frac{{\partial }^{2}U}{\partial {\text{λ}}^2}|}_{\text{λ}=F^{\star }(x)}{g}_2(x)+I\\ r_{12}(x)=g_2^T(x){\frac{{\partial }^{2}U}{\partial {\text{λ}}^2}|}_{\text{λ}=F^{\star }(x)}{g}_1(x)\\ r_{21}(x)=g_1^T(x){\frac{{\partial }^{2}W}{\partial {\text{λ}}^2}|}_{\text{λ}=F^{\star }(x)}{g}_2(x)\\ r_{22}(x)={\gamma }^{2}I+g_1^T(x){\frac{{\partial }^{2}EW}{\partial {\text{λ}}^2}|}_{\text{λ}=F^{\star }(x)}{g}_1(x).\end{array}\}$

(11.39)

The following theorem then presents sufficient conditions for the solvability of the finite-horizon problem.

Theorem 11.2.1 Consider the discrete-time nonlinear system (11.29), and the finite-horizon DSFBMH2HINLCP with cost functionals (11.32), (11.33). Suppose there exists a pair of negative and positive-definite C² (with respect to the first argument)-functions $W,U:M\times Z\to \Re$ locally defined in a neighborhood M of the origin x = 0, such that W(0,k) = 0 and U(0,k) = 0, and satisfying the coupled DHJIEs:

$\begin{array}{l}W(x,k)=W(f(x)+g_1(x)w^{\star }+g_2(x)u^{\star },k+1)+\frac{1}{2}({\gamma }^2{\Vert w\Vert }^2-{\Vert z_k^{\star }\Vert }^2),\\ W(x,K+1)=0,\end{array}$

(11.40)

$\begin{array}{l}U(x,k)=U(f(x)+g_1(x)w^{\star }+g_2(x)u^{\star },k+1)+\frac{1}{2}{\Vert z_k^{\star }\Vert }^2,\\ U(x,K+1)=0,\end{array}$

(11.41)

together with the conditions

$r_{22}(0)>0,\det [r_{11}(0)-r_{22}^{-1}(0)r_{21}(0)]\ne 0.$

(11.42)

Then the state-feedback controls defined implicitly by

$w^{\star }=-g_1^T(x)\frac{1}{{\gamma }^2}{\frac{\partial W}{\partial \text{λ}}|}_{\text{λ=}f(x)+g_1(x)w^{\star }+g_2(x)u^{\star }}$

(11.43)

$u^{\star }=-g_2^T(x)\frac{1}{{\gamma }^2}{\frac{\partial U}{\partial \text{λ}}|}_{\text{λ=}f(x)+g_1(x)w^{\star }+g_2(x)u^{\star }}$

(11.44)

solve the finite-horizon DSFBMH2HINLCP for the system. Moreover, the optimal costs are given by

$J_{1d}^{\star }(u^{\star },w^{\star })=W(k_0,x_0),$

(11.45)

$J_{2d}^{\star }(u^{\star },w^{\star })=U(k_0,x_0).$

(11.46)

Proof: We prove item (a) in Definition 11.2.1 first. Assume there exist solutions W < 0, U > 0 of the DHJIEs (11.40), (11.41), and consider the Hamiltonian functions H₁(.,.,.), H₂(.,.,.). Applying the necessary conditions for optimality

$\frac{\partial H_1}{\partial w}(x,u,w)=0,\frac{\partial H_2}{\partial u}(x,u,w)=0,$

and solving these for w^⋆, u^* respectively, we get the Nash-equilibrium strategies (11.43), (11.44). Moreover, if the conditions (11.42) are satisfied, then the matrix

$\begin{array}{l}{{\partial }^{2}H|}_{w=0,u=0}(0)=\left[\begin{array}{cc}{r}_{11}(0)& r_{12}(0)\\ r_{21}(0)& r_{22}(0)\end{array}\right]=\\ \left[\begin{array}{cc}I& r_{12}(0)r_{22}^{-1}(0)\\ 0& I\end{array}\right]\left[\begin{array}{cc}{r}_{11}(0)-r_{12}(0)r_{22}^{-1}(0)r_{21}(0)& 0\\ 0& r_{22}(0)\end{array}\right]\left[\begin{array}{cc}I& 0\\ r_{22}^{-1}(0)r_{21}(0)& I\end{array}\right]\end{array}$

is nonsingular. Therefore, by the Implicit-function Theorem, there exist open neighborhoods X₁ of x = 0, W₁ of w = 0 and U₁ of u = 0, such that the equations (11.43), (11.44) have unique solutions.

Now suppose, (u^*, w^⋆ ) have been obtained from (11.43), (11.44), then subsituting in the DHJIEs (11.34), (11.35) yield the DHJIEs (11.40), (11.41). Moreover, by Taylor-series expansion, we can write

$\begin{array}{l}{H}_1(x,w,u^{\star }(x),W)=H_1(x,w^{\star }(x).u^{\star }(x),W)+\frac{1}{2}{\left(w-w^{\star }\left(x\right)\right)}^T[r_{22}(x)+\\ O\left(\Vert w-w^{\star }(x)\Vert \right)]\left(w-w^{\star }\left(x\right)\right).\end{array}$

In addition, since r₂₂(0) > 0 implies r₂₂(x) > 0 for all x in a neighborhood X₂ of x = 0 by the Inverse-function Theorem [234], it then follows from above that there exists also a neighborhood W₂ of w = 0 such that

$\begin{array}{l}{H}_1(x,w,u^{\star }(x),W)\ge H_1(x,w^{\star }(x),u^{\star }(x),W)=0\forall x\in X_2,\forall w\in W_2,\\ \iff W(f(x)+g_1(x)w+g_2(x)u^{\star }(x),k+1)-W(x,k)+\frac{1}{2}({\gamma }^2{\Vert w\Vert }^2-{\Vert u^{\star }(x)\Vert }^2-{\Vert h_1\Vert }^2)\ge 0\\ \forall x\in X_2,\forall w\in W_2.\end{array}$

Setting now w = 0, we have

$\tilde{W}(f(x)+g_2(x)u^{\star }(x),k+1)-\tilde{W}(x,k)\le -\frac{1}{2}{\Vert u^{\star }(x)\Vert }^2-\frac{1}{2}{\Vert h_1(x)\Vert }^2\le 0$

for some function $\tilde{W}$ = − W > 0. Hence, the closed-loop system is Lyapunov-stable. To prove item (b), consider the Hamiltonian function H₂ (., w^⋆, ., .) and expand it in Taylor’sseries:

$H_2(x,w^{\star },u,U)=H_2(x,w^{\star },u^{\star },U)+\frac{1}{2}{\left(u-u^{\star }\right)}^T\left[r_{11}(x)+O\left(\Vert u-u^{\star }\Vert \right)\right]\left(u-u^{\star }\right).$

Since

$r_{11}(0)=I+g_2^T(0)\frac{{\partial }^{2}W}{\partial {\text{λ}}^2}(0)g_2(0)\ge I,$

again there exists a neighborhood ${\tilde{X}}_2$ of x = 0 such that r₁₁(x) > 0 by the Inverse-function Theorem. Therefore,

$H_2(x,w^{\star },u,U)\ge H_2(x,w^{\star },u^{\star },U)=0\forall u\in \mathcal{U}$

and the H₂-cost is minimized.

Finally, we determine the optimal costs of the strategies. For this, consider the cost functional J_1d(u^*, w^⋆ ) and write it as

$\begin{array}{l}{J}_{1d}(u,w)+W(x_{k+1},K+1)-W(x_{k_0},k_0)={\displaystyle \sum _{k=k_0}^K\{\frac{1}{2}(}{\gamma }^2{\Vert w_k^{\star }\Vert }^2-{\Vert z_k^{\star }\Vert }^2)+\\ W(x_{k+1},K+1)-W(x_k,k)\}\\ ={\displaystyle \sum _{k=k_0}^K{H}_1(x,w^{\star },u^{\star },W)}=0.\end{array}$

Since $W(x_{k+1},K+1)=0$, we have the result. Similarly, for J₂(w^⋆, u^* ), we have

$\begin{array}{l}{J}_{2d}(u,w)+U(x_{k+1},K+1)-U(x_{k_0},k_0)={\displaystyle \sum _{k=k_0}^K\left\{\frac{1}{2}{\Vert z_k^{\star }\Vert }^2+U(x_{k+1},K+1)-U(x_k,k)\right\}}\\ ={\displaystyle \sum _{k=k_0}^K{H}_2(x,w^{\star },u^{\star },U)}=0.\end{array}$

and since U(x_K+1, K + 1) = 0, the result also follows. □

The above result can be specialized to the linear discrete-time system

${\sum }^{dl}:\{\begin{array}{l}{\dot{x}}_{k+1}=A{x}_1+B_1{w}_k+B_2{u}_k\\ z_k=C_1{x}_k+D_{12}{w}_k\\ y_k=x_k\end{array}$

(11.47)

where all the variables and matrices have their previous meanings and dimensions. Then we have the following corollary to Theorem 11.2.1.

Corollary 11.2.1 Consider the linear system Σ^dl under the Assumption 11.1.2. Suppose there exist P_1,k < 0 and P_2,k > 0 symmetric solutions of the cross-coupled discrete-Riccati difference-equations (DRDEs):

$\begin{array}{l}{P}_{1,k}=A^T\{P_{1,k}-2P_{1,k}{B}_1{B}_{\text{γ,}k}^{-1}{\text{Γ}}_{1,k}-2P_{1,k}{B}_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}+\\ 2{\text{Γ}}_{1,k}^T{B}_{\text{γ,}k}^{-T}{B}_1{P}_1{B}_2{\mathrm{\Lambda }}_k^{-1}{B}_2{P}_{2,k}{\text{Γ}}_{2,k}+{\text{Γ}}_{1,k}^T{B}_{\text{γ,}k}^{-1}{B}_1{P}_1{B}_1{B}_{\text{γ,}k}^{-1}{\text{Γ}}_{1,k}+\\ {\text{Γ}}_{2,k}^T{P}_2{B}_2{\mathrm{\Lambda }}_k^{-T}{B}_2{P}_{1,k}{B}_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}{\text{Γ}}_{2,k}^T+{\gamma }^2{\text{Γ}}_{1,k}^T{B}_{\text{γ,}k}^{-T}{B}_{\text{γ,}k}^{-1}{\text{Γ}}_{1,k}^T-\\ {\text{Γ}}_{2,k}^T{P}_{2,k}{B}_2{\mathrm{\Lambda }}_k^{-T}{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_2{\text{Γ}}_{2,k}\}A-C_1^T{C}_1,P_{1,{\rm K}}=0\end{array}$

(11.48)

$\begin{array}{l}{P}_{2,k}=A^T\{P_{2,k}-2P_{2,k}{B}_1{B}_{\text{γ,}k}^{-1}{\text{Γ}}_{1,k}-2P_{2,k}{B}_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}{\text{Γ}}_{2,k}+\\ 2{\text{Γ}}_{1,k}^T{B}_{\text{γ,}k}^{-T}{B}_1{P}_{2,k}{B}_2{\mathrm{\Lambda }}_k^{-1}{B}_{2,k}^T{P}_{2,k}{\text{Γ}}_{2,k}+\\ {\text{Γ}}_{2,k}^T{P}_{2,k}{B}_2{\mathrm{\Lambda }}_k^{-T}{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}{\text{Γ}}_{2,k}\}A+C_1^T{C}_1,P_{2,{\rm K}}=0\end{array}$

(11.49)

$B_{\text{γ,k}}:=[{\gamma }^{2}I-B_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}{B}_1+B_1^T{P}_{1,k}{B}_1]>0$

(11.50)

for all k in [k₀, K]. Then, the Nash-equilibrium strategies uniquely specified by

$w_{l,k}^{\star }=-B_{\text{γ,k}}^{-1}{\text{Γ}}_{1,k}A{x}_k,k\in [k_0,K]$

(11.51)

$u_{l,k}^{\star }=-{\mathrm{\Lambda }}_k^{-1}{B}_{\text{2}}^T{P}_{2,k}{\text{Γ}}_{2,k}A{x}_k,k\in [k_0,K]$

(11.52)

where

$\begin{array}{l}{\mathrm{\Lambda }}_k:=(I+B_{\text{2}}^T{P}_{2,k}{B}_2),\forall k,\\ {\text{Γ}}_{1,k}:=[B_1^T{P}_{1,k}-B_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}]\forall k,\\ {\text{Γ}}_{2,k}:=[I-B_1{B}_{\text{γ,k}}^{-1}(B_1^T{P}_{1,k}-B_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k})]\forall k,\end{array}$

solve the finite-horizon DSFBMH2HINLCP for the system. Moreover, the optimal costs for the game are given by

$J_{1,l}(u^{\star },w^{\star })=\frac{1}{2}{x}_{k_0}^T{P}_{1,k_0}{x}_{k_0},$

(11.53)

$J_{2,l}(u^{\star },w^{\star })=\frac{1}{2}{x}_{k_0}^T{P}_{2,k_0}{x}_{k_0}.$

(11.54)

Proof: Assume the solutions to the coupled HJIEs are of the form,

$W(x_k,k)=\frac{1}{2}{x}_k^T{P}_{1,k}{x}_k,P_{1,k}<0,k=1,\mathrm{\dots },K,$

(11.55)

$U(x_k,k)=\frac{1}{2}{x}_k^T{P}_{2,k}{x}_k,P_{2,k}>0,k=1,\mathrm{\dots },K.$

(11.56)

Then, the Hamiltonians H₁(., ., ., .), H₂(., ., ., .) are given by

$\begin{array}{l}{H}_{1,l}\left(x,w,u,W\right)=\frac{1}{2}{\left(Ax+B_{1}w+B_{2}u\right)}^T{P}_{1,k}\left(Ax+B_{1}w+B_{2}u\right)-\\ \frac{1}{2}{x}^T{p}_{1,k}x+\frac{1}{2}{\gamma }^2{\Vert w\Vert }^2-\frac{1}{2}{\Vert z\Vert }^2,\end{array}$

$\begin{array}{l}{H}_{2,l}\left(x,w,u,U\right)=\frac{1}{2}{\left(Ax+B_{1}w+B_{2}u\right)}^T{P}_{2,k}\left(Ax+B_{1}w+B_{2}u\right)-\\ \frac{1}{2}{x}^T{p}_{2,k}x+\frac{1}{2}{\Vert z\Vert }^2.\end{array}$

Applying the necessary conditions for optimality, we get

$\frac{\partial H_{1,l}}{\partial w}=B_1^T{P}_{1,k}(Ax+B_{1}w+B_{2}u)+{\gamma }^{2}w=0,$

(11.57)

$\frac{\partial H_{2,l}}{\partial u}=B_2^T{P}_{2,k}(Ax+B_{1}w+B_{2}u)+u=0.$

(11.58)

Solving the last equation for u we have

$u=-{\left(I+B_2^T{P}_{2,k}{B}_2\right)}^{-1}{B}_2^T{P}_{2,k}\left(Ax+B_{1}w\right),$

which upon substitution in the first equation gives

$\begin{array}{l}{B}_1^T{P}_{1,k}\{Ax+B_{1}w-B_2{\left(I+B_2^T{P}_{2,k}{B}_2\right)}^{-1}{B}_2^T{P}_{2,k}(Ax+B_{1}w)\}+{\gamma }^{2}w=0\\ \iff w_{l,k}^{\star }=-B_{\text{γ,k}}^{-1}[B_1^T{P}_{1,k}-B_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}]A{x}_k\\ =-B_{\text{γ,k}}^{-1}{\text{Γ}}_{1,k}A{x}_{k}k\in [k_0,K],\end{array}$

(11.59)

if and only if

$B_{\text{γ,k}}:=[{\gamma }^{2}I-B_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}{B}_1+B_1^T{P}_{1,k}{B}_1]>0\forall k,$

where

$\begin{array}{l}{\mathrm{\Lambda }}_k:=(I+B_2^T{P}_{2,k}{B}_2),\forall k,\\ {\text{Γ}}_{1,k}:=[B_1^T{P}_{1,k}-B_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}]\forall k.\end{array}$

Notice that Λ_k is nonsingular for all k since P_2,k is positive-definite. Now, substitute w^⋆ in the expression for u to get

$\begin{array}{l}{u}_{l,k}^{\star }={\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}[I-B_1{B}_{\text{γ,k}}^{-1}(B_1^T{P}_{1,k}-B_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k})]A{x}_k,k\in [k_0,K]\\ =-{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k}{\text{Γ}}_{2,k}A{x}_k\end{array}$

(11.60)

where

${\text{Γ}}_{2,k}:=[I-B_1{B}_{\text{γ,k}}^{-1}(B_1^T{P}_{1,k}-B_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{P}_{2,k})].$

Finally, substituting $(u_{l,k}^{\star },w_{l,k}^{\star })$ in the DHJIEs (11.40), (11.41), we get the DRDEs (11.48), (11.49). The optimal costs are also obtained by substitution in (11.45), (11.46). □

Remark 11.2.1 Note, in the above Corollary 11.2.1 for the solution of the linear discrete-time problem, it is better to consider strictly positive-definite solutions of the DRDEs (11.48), (11.49) because the condition B _γ,k > 0 must be respected for all k.

11.2.1  The Infinite-Horizon Problem

In this subsection, we consider similarly the infinite-horizon DSFBMH2HINLCP for the affine discrete-time nonlinear system Σ^da. We let K → ∞, and seek time-invariant functions and feedback gains that solve the DSFBMH2HINLCP. Again we require that the closed-loop system be locally asymptotically-stable, and for this, we need the following definition of detectability for the discrete-time system Σ^da.

Definition 11.2.2 The pair {f,h} is said to be locally zero-state detectable if there exists a neighborhood $\tilde{\mathcal{O}}$ of x = 0 such that, if x_k is a trajectory of x_k+1 = f(x_k) satisfying $x(k_0)\in \tilde{\mathcal{O}}$, then h(x_k) is defined for all k ≥ k₀, and h(x_k) = 0 for all k ≥ k_s, implies ${\lim }_{k\to \infty }{x}_k=\text{}\text{}0$. Moreover {f,h} is said to be zero-state detectable if $\tilde{\mathcal{O}}=\mathcal{X}$.

Theorem 11.2.2 Consider the nonlinear system Σ^da defined by (11.29) and the infinite-horizon DSFBMH2HINLCP with cost functionals (11.32), (11.8). Suppose

(H1) the pair {f, h₁ } is zero-state detectable;

(H2) there exists a pair of negative and positive-definite C²-functions $\tilde{W},\tilde{U}:\tilde{M}\times Z\to \Re$ locally defined in a neighborhood $\tilde{M}$ of the origin x = 0, such that $\tilde{W}(0)=0\text{and}\tilde{U}(0)=0$, and satisfying the coupled DHJIEs:

$\tilde{W}(f(x)+g_1(x)w^{\star }+g_2(x)u^{\star })-\tilde{W}(x)+\frac{1}{2}({\gamma }^2{\Vert w\Vert }^2-{\Vert z_k^{\star }\Vert }^2)=0,W(0)=0,$

(11.61)

$\tilde{U}(f(x)+g_1(x)w^{\star }+g_2(x)u^{\star })-\tilde{U}(x)+\frac{1}{2}{\Vert z_k^{\star }\Vert }^2=0,U(0)=0.$

(11.62)

together with the conditions

$r_{22}(0)>0,\det [r_{11}(0)-r_{22}^{-1}(0)r_{21}(0)]\ne 0.$

(11.63)

Then, the state-feedback controls defined implicitly by

$w^{\star }=-g_1^T(x)\frac{1}{{\gamma }^2}{\frac{\partial \tilde{W}}{\partial \text{λ}}|}_{\text{λ=}f(x)+g_1(x)w^{\star }+g_2(x)u^{\star }}$

(11.64)

$u^{\star }=-g_2^T(x){\frac{\partial \tilde{U}}{\partial \text{λ}}|}_{\text{λ=}f(x)+g_1(x)w^{\star }+g_2(x)u^{\star }}$

(11.65)

solve the infinite-horizon DSFBMH2HINLCP for the system. Moreover, the optimal costs are given by

$J_{1d}^{\star }(u^{\star },w^{\star })=\tilde{W}(x_0),$

(11.66)

$J_{2d}^{\star }(u^{\star },w^{\star })=\tilde{U}(x_0).$

(11.67)

Proof: We only prove item (c) in the definition, since the proofs of items (a) and (b) are exactly similar to the finite-horizon problem. Accordingly, using similar manipulations as in the proof of item (a) of Theorem 11.2.1, it can be shown that with w ≡ 0,

$\tilde{W}(f(x)+g_2(x)u^{\star })-\tilde{W}(x)=-\frac{1}{2}{\Vert z\Vert }^2.$

Therefore, the closed-loop system is Lyapunov-stable. Further, the condition $\tilde{W}(f(x)+g_2(x)u^{\star }(x))\equiv \tilde{W}(x)\forall k\ge k_c$, for some k_c ≥ k₀, implies that u^* ≡ 0, h₁(x) ≡ 0 ∀ k ≥ k_c. By hypothesis (H1), this implies lim_t→∞ x_k = 0, and by LaSalle’s invariance-principle, we conclude asymptotic-stability. □

The above theorem can again be specialized to the linear system Σ^dl in the following corollary.

Corollary 11.2.2 Consider the discrete linear system Σ^dl under the Assumption 11.1.2. Suppose there exist ${\overline{P}}_1<0and{\overline{P}}_2>0$ symmetric solutions of the cross-coupled discrete- algebraic Riccati equations (DAREs):

$\begin{array}{l}{\overline{P}}_1=A^T\{{\overline{P}}_1-2{\overline{P}}_1{B}_1{B}_{\gamma }^{-1}{\text{Γ}}_1-2{\overline{P}}_1{B}_2{\mathrm{\Lambda }}^{-1}{B}_2^T{\overline{P}}_2+2{\text{Γ}}_1^T{B}_{\gamma }^{-T}{B}_1{\overline{P}}_1{B}_2{\mathrm{\Lambda }}^{-1}{B}_2{\overline{P}}_2{\text{Γ}}_2+\\ {\text{Γ}}_1^T{B}_{\gamma }^{-1}{B}_1{\overline{P}}_1{B}_1{B}_{\gamma }^{-1}{\text{Γ}}_1+{\text{Γ}}_2^T{\overline{P}}_2{B}_2{\mathrm{\Lambda }}^{-T}{B}_2{\overline{P}}_1{B}_2{\mathrm{\Lambda }}^{-1}{B}_2^T{\overline{P}}_2{\text{Γ}}_2+{\gamma }^2{\text{Γ}}_1^T{B}_{\gamma }^{-T}{B}_{\gamma }^{-1}{\text{Γ}}_1^T-\\ {\text{Γ}}_2^T{\overline{P}}_2{B}_2{\mathrm{\Lambda }}^{-T}{\mathrm{\Lambda }}^{-1}{B}_2^T{\overline{P}}_2{\text{Γ}}_2\}A-C_1^T{C}_1,\end{array}$

(11.68)

$\begin{array}{l}{\overline{P}}_2=A^T\{{\overline{P}}_2-2{\overline{P}}_2{B}_1{B}_{\gamma }^{-1}{\text{Γ}}_1-2{\overline{P}}_2{B}_2{\mathrm{\Lambda }}^{-1}{B}_2^T{\overline{P}}_2{\text{Γ}}_2+\\ 2{\text{Γ}}_1^T{B}_{\gamma }^{-T}{B}_1{\overline{P}}_2{B}_2{\mathrm{\Lambda }}_k^{-1}{B}_2^T{\overline{P}}_2{\text{Γ}}_2+{\text{Γ}}_2^T{\overline{P}}_2{B}_2{\mathrm{\Lambda }}^{-1}{\mathrm{\Lambda }}^{-1}{B}_2^T{\overline{P}}_2{\text{Γ}}_2\}A+C_1^T{C}_1.\end{array}$

(11.69)

$B_{\gamma }:=[{\gamma }^{2}I-B_2{\mathrm{\Lambda }}^{-1}{B}_2^T{P}_2{B}_1+B_1^T{P}_1{B}_1]>0$

(11.70)

Then the Nash-equilibrium strategies uniquely specified by

$w_{l,k}^{\star }=-B_{\gamma }^{-1}{\text{Γ}}_{1}A{x}_k,$

(11.71)

$u_{l,k}^{\star }=-{\mathrm{\Lambda }}^{-1}{B}_2^T{\overline{P}}_2{\text{Γ}}_{2}A{x}_k,$

(11.72)

where

$\begin{array}{l}\mathrm{\Lambda }:=(I+B_2^T{\overline{P}}_2{B}_2),\\ {\text{Γ}}_1:=[B_1^T{\overline{P}}_1-B_2{\mathrm{\Lambda }}^{-1}{B}_2^T{P}_2],\\ {\text{Γ}}_2:=[I-B_1{B}_{\gamma }^{-1}(B_1^T{P}_1-B_2{\mathrm{\Lambda }}^{-1}{B}_2^T{\overline{P}}_2)],\end{array}$

solve the infinite-horizon DSF BMH2HINLCP for the system. Moreover, the optimal costs for the game are given by

$J_{1,l}(u^{\star },w^{\star })=\frac{1}{2}{x}_{k_0}^T{\overline{P}}_1{x}_{k_0},$

(11.73)

$J_{2,l}(u^{\star },w^{\star })=\frac{1}{2}{x}_{k_0}^T{\overline{P}}_2{x}_{k_0}.$

(11.74)

Proof: Take

$\begin{array}{l}Y(x_k)=\frac{1}{2}{x}_k^T{\overline{P}}_1{x}_k,{\overline{P}}_1<0\\ V(x)=\frac{1}{2}{x}_k^T{\overline{P}}_2{x}_k,{\overline{P}}_2>0\end{array}$

and apply the results of the theorem. □

11.3  Extension to a General Class of Discrete-Time Nonlinear Systems

In this subsection, we similarly extend the results of the previous subsection to a more general class of nonlinear discrete-time systems which is not necessarily affine. We consider the following state-space model defined on $\mathcal{X}\subset {\Re }^n$ in local coordinates (x₁,…, x_n)

$\sum :\{\begin{array}{l}{\dot{x}}_{k+1}=\tilde{F}(x_k,w_k,u_k),x(t_0)=x_0\\ z_k=\tilde{Z}(x_k,u_k)\\ y_k=x_k,\end{array}$

(11.75)

where all the variables have their previous meanings, while $\tilde{F}:\mathcal{X}\times \mathcal{W}\times \mathcal{U}\to \mathcal{X},\tilde{Z}:\mathcal{X}\times \mathcal{U}\to {\Re }^s$ are smooth functions of their arguments. In addition, we assume that $\tilde{F}(0,0,0)=0\text{and}\tilde{\text{Z}}\text{(0,0)=0}$. Furthermore, define similarly the Hamiltonian functions corresponding to the cost functionals (11.32), (11.33), ${\tilde{K}}_i:\mathcal{X}\times \mathcal{W}\times \mathcal{U}\times \Re \to \Re ,i=1,2$ respectively:

${\tilde{K}}_1(x,w,u,\tilde{W})=\tilde{W}(\tilde{F}(x,w,u))-\tilde{W}(x)+\frac{1}{2}{\gamma }^2{\Vert w\Vert }^2-{\Vert \tilde{z}(x,u)\Vert }^2,$

${\tilde{K}}_2(x,w,u,\overline{U})=\tilde{U}(\tilde{F}(x,w,u))-\tilde{U}(x)+\frac{1}{2}{\Vert \tilde{z}(x,u)\Vert }^2,$

for some smooth functions $\tilde{W},\tilde{U}:\mathcal{X}\to \Re$. In addition, define also

${\partial }^2\tilde{K}(x)\triangleq \left[\begin{array}{cc}\frac{{\partial }^2{\tilde{K}}_2}{\partial u^2}& \frac{{\partial }^2{\tilde{K}}_2}{\partial w\partial u}\\ \frac{{\partial }^2{\tilde{K}}_1}{\partial u\partial w}& \frac{{\partial }^2{\tilde{K}}_1}{\partial w^2}\end{array}\right](x)=\left[\begin{array}{cc}{s}_{11}(x)& s_{12}(x)\\ s_{21}(x)& S_{22}(x)\end{array}\right],$

where

$s_{11}(0)={\left[{\left(\frac{\partial \tilde{F}}{\partial u}\right)}^T\frac{{\partial }^2\tilde{U}}{\partial {\text{λ}}^2}(0)\frac{\partial \tilde{F}}{\partial u}+{\left(\frac{\partial \tilde{Z}}{\partial u}\right)}^T\frac{\partial \tilde{Z}}{\partial u}\right]}_{x=0,w=0,u=0},$

$s_{12}(0)={\left[{\left(\frac{\partial \tilde{F}}{\partial u}\right)}^T\frac{{\partial }^2\tilde{U}}{\partial {\text{λ}}^2}(0)\frac{\partial \tilde{F}}{\partial w}\right]}_{x=0,w=0,u=0},$

$s_{21}(0)={\left[{\left(\frac{\partial \tilde{F}}{\partial w}\right)}^T\frac{{\partial }^2\tilde{W}}{\partial {\text{λ}}^2}(0)\frac{\partial \tilde{F}}{\partial u}\right]}_{x=0,w=0,u=0},$

$s_{22}(0)={\left[{\left(\frac{\partial \tilde{F}}{\partial w}\right)}^T\frac{{\partial }^2\tilde{W}}{\partial {\text{λ}}^2}(0)\frac{\partial \tilde{F}}{\partial w}+{\gamma }^{2}I\right]}_{x=0,w=0,u=0}.$

We then make the following assumption.

Assumption 11.3.1 For the Hamiltonian functions, ${\tilde{K}}_1,{\tilde{K}}_2$, we assume

$s_{22}(0)>0,\det [s_{11}(0)-s_{12}(0)s_{22}^{-1}{s}_{21}(0)]\ne 0.$

Under the above assumption, the Hessian matrix ${\partial }^2\tilde{K}(0)$ is nonsingular, and therefore by the Implicit-function Theorem, there exists an open neighborhood M₀ of x = 0 such that the equations

$\begin{array}{l}\frac{\partial {\tilde{K}}_1}{\partial w}(x,{\tilde{w}}^{\star }(x),{\tilde{u}}^{\star }(x))=0,\\ \frac{\partial {\tilde{K}}_2}{\partial u}(x,{\tilde{w}}^{\star }(x),{\tilde{u}}^{\star }(x))=0\end{array}$

have unique solutions ${\tilde{u}}^{\star }(x),{\tilde{w}}^{\star }(x),\text{with}{\tilde{u}}^{\star }(0)=0,{\tilde{w}}^{\star }(0)=0$. Moreover, the pair $({\tilde{u}}^{\star },{\tilde{w}}^{\star })$ constitutes a Nash-equilibrium solution to the dynamic game (11.32), (11.33), (11.75). The following theorem then summarizes the solution to the infinite-horizon problem for the general class of discrete-time nonlinear systems (11.75).

Theorem 11.3.1 Consider the discrete-time nonlinear system (11.75) and the DSFBMH2HINLCP for this system. Suppose Assumption 11.3.1 holds, and also the following:

(Ad1) the pair {$\tilde{F}(x,0,0),\tilde{\text{Z}}\text{(}x\text{,0)}$} is zero-state detectable;

(Ad2) there exists a pair of C² locally negative and positive-definite functions $\tilde{W},\tilde{U}:\tilde{M}\to \Re$ respectively, defined in a neighborhood $\tilde{M}$ of x = 0, vanishing at x = 0 and satisfying the pair of coupled DHJIEs:

$\tilde{W}(\tilde{F}(x,{\tilde{w}}^{\star }(x),{\tilde{u}}^{\star }(x)))-\tilde{W}(x)+\frac{1}{2}{\gamma }^2{\Vert {\tilde{w}}^{\star }(x)\Vert }^2-\frac{1}{2}{\Vert \tilde{Z}(x,{\tilde{u}}^{\star }(x))\Vert }^2=0,\tilde{W}(0)=0,$

$\tilde{U}(\tilde{F}(x,{\tilde{w}}^{\star }(x),{\tilde{u}}^{\star }(x)))-\tilde{U}(x)+\frac{1}{2}{\Vert \tilde{Z}(x,{\tilde{u}}^{\star }(x))\Vert }^2=0,\tilde{U}(0)=0;$

(A3) the pair $\left\{\tilde{F}(x,{\tilde{w}}^{\star }(x),0),\tilde{Z}(x,0)\right\}$ is locally zero-state detectable.

Then the state-feedback controls (${\tilde{u}}^{\star }(x),{\tilde{w}}^{\star }(x)$) solve the dynamic game problem and the DSFBMH2HINLCP for the system (11.75). Moreover, the optimal costs of the policies are given by

$\begin{array}{l}{\tilde{J}}_{1d}^{\star }({\tilde{w}}^{\star },{\tilde{u}}^{\star })=\tilde{W}(x_0),\\ {\tilde{J}}_{2d}^{\star }({\tilde{w}}^{\star },{\tilde{u}}^{\star })=\tilde{U}(x_0).\end{array}$

Proof: The proof can be pursued along the same lines as the previous results. □

11.4  Notes and Bibliography

This chapter is mainly based on the paper by Lin [180]. The approach adopted throughout the chapter was originally inspired by the paper by Limebeer et al. [179] for linear systems. The chapter mainly extended the results of the paper to the nonlinear case. But in addition, the discrete-time problem has also been developed. Finally, application of the results to tracking control for Robot manipulators can be found in [80].