13

Solving the Hamilton-Jacobi Equation

In this chapter, we discuss some approaches for solving the Hamilton-Jacobi-equations (HJE) associated with the optimal control problems for affine nonlinear systems discussed in this book. This has been the biggest bottle-neck in the practical application of nonlinear ${\mathcal{H}}_{\infty }$-control theory. There is no systematic numerical approach for solving the HJIEs. Various attempts have however been made in this direction in the past three decades. Starting with the work of Lukes [193], Glad [112], who proposed a polynomial approximation approach, Van der Schaft [264] applied the approach to the HJIEs and developed a recursive approach which was also refined by Isidori [145]. Since then, many other authors have proposed similar approaches for solving the HJEs [63, 80, 125, 260, 286]. However, all the contributions so far in the literature are local.

The main draw-backs with the above approaches for solving the HJIE are that, (i) they are not closed-form, and convergence of the sequence of solutions to a closed-form solution cannot be guaranteed; (ii) there are no efficient methods for checking the positivedefiniteness of the solution; (iii) they are sensitive to uncertainties and pertubations in the system; and (iv) also sensitive to the initial condition. Thus, the global asymptotic stability of the closed-loop system cannot be guaranteed.

Therefore, more refined solutions that will guarantee global asymptotic-stability are required if the theory of nonlinear ${\mathcal{H}}_{\infty }$-control is to yield any fruits. Thus, with this in mind, more recently, Isidori and Lin [145] have shown that starting from a solution of an algebraic-Riccati-equation (ARE) related to the linear ${\mathcal{H}}_{\infty }$ problem, if one is free to choose a state-dependent weight of control input, it is possible to construct a global solution to the HJIE for a class of nonlinear systems in strict-feedback form. A parallel approach using a backstepping procedure and inverse-optimality has also been proposed in [96], and in [167]-[169] for a class of strict-feedback systems.

In this chapter, we shall review some major approaches for solving the HJE, and present one algorithm that may yield global solutions. The chapter is organized as follows. In Section 13.1, we review some popular polynomial and Taylor-series approximation methods for solving the HJEs. Then in Section 13.2, we discuss a factorization approach which may yield exact and global solutions. The extension of this approach to Hamiltonian mechanical systems is then discussed in Section 13.3. Examples are given throughout to illustrate the usefulness of the various methods.

13.1  Review of Some Approaches for Solving the HJBE/HJIE

In this section, we review some major approaches for solving the HJBE and HJIE that have been proposed in the literature. In [193], a recursive procedure for the HJBE for a class of nonlinear systems is proposed. This was further refined and generalized for affine nonlinear systems by Glad [112]. To summarize the method briefly, we consider the following affine nonlinear system

${\mathrm{\Sigma }}_a:\dot{x}\left(t\right)=a\left(x\right)+b\left(x\right)u$

(13.1)

under the quadratic cost functional:

$\min {\displaystyle {\int }_0^{\infty }\left[l\left(x\right)+\frac{1}{2}{u}^{T}Ru\right]dt},$

(13.2)

where a : ℜⁿ →ℜⁿ, b : ℜⁿ →ℜ^n×k, l : ℜⁿ →ℜ, l ≥ 0, a,b,l ∈ C^∞, 0 < R ∈ℜ^k×k. Then, the HJBE corresponding to the above optimal control problem is given by

${\text{υ}}_x\left(x\right)a\left(x\right)-\frac{1}{2}{\text{υ}}_x\left(x\right)b\left(x\right)R^{-1}{b}^T\left(x\right){\upsilon }_x^T\left(x\right)+l\left(x\right)=0,\text{υ}\left(0\right)=0,$

(13.3)

for some smooth C²(ℜⁿ) positive-semidefinite function v : ℜⁿ →ℜ, and the optimal control is given by

$u^{\star }=k\left(x\right)=-R^{-1}{b}^T\left(x\right){\text{υ}}_x^T\left(x\right).$

Further, suppose there exists a positive-semidefinite solution to the algebraic-Riccati-equation (ARE) corresponding to the linearization of the system (13.1), then it can be shown that there exists a real analytic solution v to the HJBE (13.3) in a neighborhood $\mathcal{O}$ of the origin [193]. Therefore, if we write the linearization of the system and the cost function as

$\begin{array}{l}l\left(x\right)=\frac{1}{2}{x}^{T}Qx+l_h\left(x\right)\\ a\left(x\right)=Ax+a_h\left(x\right)\\ b\left(x\right)=B+b_h\left(x\right)\\ \text{υ}\left(x\right)=\frac{1}{2}{x}^{T}Px+{\text{υ}}_h\left(x\right)\end{array}$

where $Q=\frac{\partial l}{\partial x}(0),A=\frac{\partial a}{\partial x}(0),B=b(0),P=\frac{\partial v}{\partial x}(0)\text{and}{l}_h,a_h,b_h,v_h$ contain higher-order terms. Substituting the above expressions in the HJBE (13.3), it splits into two parts:

$A^{T}P+PA-PB{R}^{-1}{B}^{T}P+Q=0$

(13.4)

${\text{υ}}_{hx}\left(x\right)A_{c}x+{\text{υ}}_x\left(x\right)a_h\left(x\right)-\frac{1}{2}{\text{υ}}_{hx}\left(x\right)B{R}^{-1}{B}^T{\text{υ}}_{hx}^T\left(x\right)-\frac{1}{2}{\text{υ}}_x\left(x\right){\beta }_h\left(x\right){\text{υ}}_x^T\left(x\right)+l_h\left(x\right)=0$

(13.5)

where

$A_c=A-B{R}^{-1}{B}^{T}P,{\beta }_h\left(x\right)=b\left(x\right)R^{-1}{b}^T\left(x\right)-B{R}^{-1}{B}^T$

and β_h contains terms of degree 1 and higher. Equation (13.4) is the ARE of linear-quadratic control for the linearized system. Thus, if the system is stabilizable and detectable, there exists a unique positive-semidefinite solution to this equation. Hence, it represents the first-order approximation to the solution of the HJBE (13.3). Now letting the superscript (m) denote the mth-order terms, (13.5) can be written as

$-{\text{υ}}_{hx}^{m+1}\left(x\right)A_{c}x={\left[{\text{υ}}_x(x)a_h\left(x\right)-\frac{1}{2}{\text{υ}}_{hx}\left(x\right)B{R}^{-1}{B}^T{\text{υ}}_{hx}^T\left(x\right)-\frac{1}{2}{\text{υ}}_x\left(x\right){\beta }_h\left(x\right){\text{υ}}_x^T\left(x\right)\right]}^m+l_h^m\left(x\right).$

(13.6)

The RHS contains only m − th, (m − 1) − th,…-order terms of v. Therefore, equation (13.5) defines a linear system of equations for the (m + 1)th order coefficients with the RHS containing previously computed terms. Thus, v^m+1 can be computed recursively from (13.6). It can also be shown that the system is nonsingular as soon as A_c is a stable matrix. This is satisfied vacuously if P is a stabilizing solution to the ARE (13.4).

We consider an example.

Example 13.1.1 Consider the second-order system

${\dot{x}}_1=-x_1^2+x_2$

(13.7)

${\dot{x}}_2=-x_2+u$

(13.8)

with the cost functional

$J={\displaystyle {\int }_0^{\infty }\frac{1}{2}\left(x_1^2+x_2^2+u^2\right)dt}.$

The solution of the ARE (13.4) gives the quadratic (or second-order) approximation to the solution of the HJBE:

$V^{\left[2\right]}\left(x\right)=x_1^2+x_1{x}_2+\frac{1}{2}{x}_2^2.$

Further, the higher-order terms are computed recursively up to fourth-order to obtain

$\begin{array}{l}V\approx V^{\left[2\right]}+V^{\left[3\right]}+V^{\left[4\right]}=x_1^2+x_1{x}_2+\frac{1}{2}{x}_2^2-1.593x_1^3-2x_1^2{x}_2-0.889x_1{x}_2^2-\\ 0.148x_2^3+1.998x_1^4+2.778x_1^3{x}_2+1.441x_1^2{x}_2^2+\\ 0.329x_1{x}_2^3+0.0288x_2^4.\end{array}$

The above algorithm has been refined by Van der Schaft [264] for the HJIE (6.16) corresponding to the state-feedback ${\mathcal{H}}_{\infty }$-control problem:

$V_x\left(x\right)f\left(x\right)+V_x\left(x\right)\left[\frac{1}{{\gamma }^2}{g}_1\left(x\right)g_1^T\left(x\right)-g_2\left(x\right)g_2^T\left(x\right)\right]V_x^T\left(x\right)+\frac{1}{2}{h}_1^T\left(x\right)h_1\left(x\right)=0,V\left(0\right)=0,$

(13.9)

for some smooth C¹-function V : M →ℜ, M ⊂ℜⁿ. Suppose there exists a solution P ≥ 0 to the ARE

$F^{T}P+PF+P\left[\frac{1}{{\gamma }^2}{G}_1{G}_1^T-G_2{G}_2^T\right]P+H_1^T{H}_1=0$

(13.10)

corresponding to the linearization of the HJIE (13.9), where $F=\frac{\partial f}{\partial x}(0),G_1=g_1(0),G_2=g_2(0),H_1=\frac{\partial h_1}{\partial x}(0).$ Let V be a solution to the HJIE, and if we write

$\begin{array}{l}V\left(x\right)=\frac{1}{2}{x}^{T}Px+V_h\left(x\right)\\ f\left(x\right)=Fx+f_h\left(x\right)\\ \frac{1}{2}\left[\frac{1}{{\gamma }^2}{g}_1\left(x\right)g_1^T\left(x\right)-g_2\left(x\right)g_2^T\left(x\right)\right]=\frac{1}{2}\left[\frac{1}{{\gamma }^2}{G}_1{G}_1^T-G_2{G}_2^T\right]+R_h\left(x\right)\\ \frac{1}{2}{h}_1^T\left(x\right)h_1\left(x\right)=\frac{1}{2}{x}^T{H}_1^T{H}_{1}x+{\theta }_h\left(x\right)\end{array}$

where V_h, f_h, R_h and θ_h contain higher-order terms. Then, similarly the HJIE (13.9) splits into two parts, (i) the ARE (13.10); and (ii) the higher-order equation

$\begin{array}{l}-\frac{\partial V_h\left(x\right)}{\partial x}{F}_{cl}x=\frac{\partial V\left(x\right)}{\partial x}{f}_h\left(x\right)+\frac{1}{2}\frac{\partial V_h\left(x\right)}{\partial x}\left[\frac{1}{{\gamma }^2}{G}_1{G}_1^T-G_2{G}_2^T\right]\frac{{\partial }^T{V}_h\left(x\right)}{\partial x}\left(x\right)+\\ \frac{1}{2}\frac{\partial V\left(x\right)}{\partial x}{R}_h\left(x\right)\frac{{\partial }^{T}V\left(x\right)}{\partial x}+{\theta }_h\left(x\right)\end{array}$

(13.11)

where

$F_{cl}\triangleq F-G_2{G}_2^{T}P+\frac{1}{{\gamma }^2}{G}_1{G}_1^{T}P.$

We can rewrite (13.11) as

$-\frac{\partial V_h^m\left(x\right)}{\partial x}{F}_{cl}x=H_m\left(x\right),$

(13.12)

where H_m(x) denotes the m-th order terms on the RHS, and thus if P ≥ 0 is a stabilizing solution of the ARE (13.10), then the above Lyapunov-equation (13.12) can be integrated for $V_h^m$ to get,

$V_h^m\left(x\right)={\displaystyle {\int }_0^{\infty }{H}_m\left(e^{F_{cl}t}x\right)dt}.$

Therefore, $V_h^m$ is determined by H_m. Consequently, since H_m depends only on $V^{(m-1)},V^{(m-2)},\mathrm{\dots },V^2=\frac{1}{2}{x}^{T}Px,V_h^m$ can be computed recursively using (13.12) and starting from V ², to obtain V similarly as

$V\approx V^{\left[2\right]}+V^{\left[3\right]}+V^{\left[4\right]}+\dots$

The above approximate approach for solving the HJBE or HJIE has a short-coming; namely, there is no guarantee that the sequence of solutions will converge to a smooth positive-semidefinite solution. Moreover, in general, the resulting solution obtained cannot be guaranteed to achieve closed-loop asymptotic-stability or global ${ℒ}_2$-gain ≤ γ since the functions f_h, R_h, θ_h are not exactly known, rather they are finite approximations from a Taylor-series expansion. The procedure can also be computationally intensive especially for the HJIE as various values of γ have to be tried. The above procedure has been refined in [145] for the measurement-feedback case, and variants of the algorithm have also been proposed in other references [63, 154, 155, 286, 260].

13.1.1  13.1.1 Solving the HJIE/HJBE Using Polynomial Expansion and Basis Functions

Two variants of the above algorithms that use basis function approximation are given in [63] using the Galerkin approximation and [260] using a series solution. In [260] the following series expansion for V is used

$V\left(x\right)=V^{\left[2\right]}\left(x\right)+V^{\left[3\right]}\left(x\right)+\dots$

(13.13)

where V ^[k] is a homogeneous-function of order k in n-scalar variables x₁, x₂,…, x_n, i.e., it is a linear combination of

$N_k^n\triangleq \left(\begin{array}{l}n+k-1\\ k\end{array}\right)$

terms of the form $x_1^{i_1}{x}_2^{i_2}\mathrm{\dots }{x}_n^{i_n},$ where i_j is a nonnegative integer for j = 1,…, n and i₁ + i₂ +. .. + i_n = k. The vector whose components consist of these terms is denoted by x^[k]; for example,

$x^{\left[1\right]}=\left[\begin{array}{l}{x}_1\\ x_2\end{array}\right],x^{\left[2\right]}=\left[\begin{array}{l}{x}_1^2\\ x_1{x}_2\\ x_2^2\end{array}\right],\mathrm{\dots .}$

To summarize the approach, rewrite HJIE (13.9) as

$V_x\left(x\right)f\left(x\right)+\frac{1}{2}{V}_x\left(x\right)G\left(x\right)R^{-1}{G}^T\left(x\right)V_x^T\left(x\right)+Q\left(x\right)=0$

(13.14)

where

$G\left(x\right)=\left[\begin{array}{cc}{g}_1\left(x\right)& g_2\left(x\right)\end{array}\right],R=\left[\begin{array}{cc}{\gamma }^2& 0\\ 0& -1\end{array}\right],Q\left(x\right)=\frac{1}{2}{x}^T{C}^{T}Cx$

and let $v={\left[\begin{array}{cc}{w}^T& u^T\end{array}\right]}^T,v_{\star }={\left[\begin{array}{cc}{w}_{\star }^T& u_{\star }^T\end{array}\right]}^T=\left[\begin{array}{cc}{l}_{\star }^T& k_{\star }^T\end{array}\right].$ Then we can expand f, G, ν as homogeneous functions:

$\begin{array}{l}f\left(x\right)=f^{\left[1\right]}\left(x\right)+f^{\left[2\right]}\left(x\right)+\dots \\ G\left(x\right)=G^{\left[0\right]}\left(x\right)+G^{\left[1\right]}\left(x\right)+\dots \\ v_{\star }\left(x\right)=v_{\star }^{\left[1\right]}\left(x\right)+v_{\star }^{\left[2\right]}\left(x\right)+\dots \end{array}$

where

$v_{\star }^{\left[k\right]}=-R^{-1}{\displaystyle \sum _{j=0}^{k-1}{\left(G^{\left[j\right]}\right)}^T{\left(V_x^{\left[k+1-j\right]}\right)}^T,}$

(13.15)

$f^{\left[1\right]}\left(x\right)=Fx,G^{\left[0\right]}=\left[\begin{array}{cc}{G}_1& G_2\end{array}\right],G_1=g_1\left(0\right),G_2=g_2\left(0\right)$. Again rewrite HJIE (13.14) as

$V_x\left(x\right)f\left(x\right)+\frac{1}{2}{v}_{\star }\left(x\right)R{v}_{\star }\left(x\right)+Q\left(x\right)=0,$

(13.16)

$v_{\star }\left(x\right)+R^{-1}{G}^T\left(x\right)V_x^T\left(x\right)=0.$

(13.17)

Substituting now the above expansions in the HJIE (13.16), (13.17) and equating terms of order m ≥ 2 to zero, we get

${\displaystyle \sum _{k=0}^{m-2}{V}_x^{\left[m-k\right]}{f}^{\left[k+1\right]}}+\frac{1}{2}{\displaystyle \sum _{k=1}^{m-1}{v}_{\star }^{\left[m-k\right]}R{v}_{\star }^{\left[k\right]}+Q^{\left[m\right]}}=0.$

(13.18)

It can be checked that, for m = 2, the above equation simplifies to

$V_x^{\left[2\right]}{f}^{\left[1\right]}+\frac{1}{2}{v}_{\star }^{\left[1\right]}R{v}_{\star }^{\left[1\right]}+Q^{\left[2\right]}=0,$

where

$f^{\left[1\right]}\left(x\right)=Ax,v_{\star }^{\left[1\right]}\left(x\right)=-R^{-1}{B}^T{V}_x^{\left[2\right]T}\left(x\right),Q^{\left[2\right]}\left(x\right)=\frac{1}{2}{x}^T{C}^{T}Cx,B=\left[\begin{array}{cc}{G}_1& G_2\end{array}\right].$

Substituting in the above equation, we obtain

$V_x^{\left[2\right]}\left(x\right)Ax+\frac{1}{2}{V}_x^{\left[2\right]}\left(x\right)B{R}^{-1}{B}^T{V}_{\mathrm{x}}^{\left[2\right]}\left(x\right)+\frac{1}{2}{x}^T{C}^{T}Cx=0.$

But this is the ARE corresponding to the linearization of the system, hence

$V^{\left[2\right]}\left(x\right)=x^{T}Px,$

where P = P ^T > 0 solves the ARE (13.10) with A_⋆ = A − BR⁻¹B^T P Hurwitz. Moreover,

$v_{\star }^{\left[1\right]}\left(x\right)=-B^{T}Px,k_{\star }^{\left[1\right]}\left(x\right)=-G_1^{T}Px.$

Consider now the case m ≥ 3, and rewrite (13.18) as

${\displaystyle \sum _{k=0}^{m-2}{V}_x^{\left[m-k\right]}{f}^{\left[k+1\right]}+\frac{1}{2}{v}_{\star }^{\left[m-1\right]T}R{v}_{\star }^{\left[1\right]}+\frac{1}{2}{\displaystyle \sum _{k=2}^{m-2}{v}_{\star }^{\left[m-k\right]T}R{v}_{\star }^{\left[k\right]}=0.}}$

(13.19)

Then, using

$v_{\star }^{\left[m-1\right]T}=-{\displaystyle \sum _{k=0}^{m-2}{V}_x^{\left[m-k\right]}{G}^{\left[k\right]}{R}^{-1}},$

equation (13.19) can be written as

$V_x^{\left[m\right]}{\overline{f}}^{\left[1\right]}=\frac{1}{2}{\displaystyle \sum _{k=1}^{m-2}{V}_x^{\left[m-k\right]}{\overline{f}}^{\left[k+1\right]}}-\frac{1}{2}{\displaystyle \sum _{k=1}^{m-2}{\left(v_{\star }^{\left[m-k\right]}\right)}^{T}R{v}_{\star }^{\left[k\right]}}$

(13.20)

where

$\overline{f}\left(x\right)\triangleq f\left(x\right)+G\left(x\right)v_{\star }^{\left[1\right]}\left(x\right).$

Equation (13.20) can now be solved for any V ^[m], m ≥ 3. If we assume every V ^[m] is of the form V ^[m] = C_mx^[m], where $C_m\in {\Re }^{1\times N_m^n}$ is a row-vector of unknown coefficients, then substituting this in (13.20), we get a system of $N_m^n$ linear equations in the unknown entries of C_m. It can be shown that if the eigenvalues of A_⋆ = A − BB^T P are nonresonant,¹ then the system of linear equations has a unique solution for all m ≥ 3. Moreover, since A_⋆ is stable, the approximation V is analytic and V ^[m] converges finitely.

To summarize the procedure, if we start with $V^{[2]}(x)=\frac{1}{2}{x}^{T}Px\text{and}{v}_{\star }^{[2]}(x)=-R^{-1}{B}^{T}Px,$ equations (13.20), (13.15) could be used to compute recursively the sequence of terms

$V^{\left[3\right]}\left(x\right),v_{\star }^{\left[2\right]}\left(x\right),V^{\left[4\right]}\left(x\right),v_{\star }^{\left[3\right]}\left(x\right),\dots$

which converges point-wise to V and ν_⋆. We consider an example of the nonlinear benchmark problem [71] to illustrate the approach.

Example 13.1.2 [260]. The system involves a cart of mass M which is constrained to translate along a straight horizontal line. The cart is connected to an inertially fixed point via a linear spring as shown in Figure 13.1. A pendulum of mass m and inertia I which rotates about a vertical line passing through the cart mass-center is also mounted. The dynamic equations for the system after suitable linearization are described by

$\begin{array}{l}\ddot{\text{ξ}}+\text{ξ}=\epsilon \left({\dot{\theta }}^2\sin \theta -\ddot{\theta }\cos \theta \right)+w\\ \ddot{\theta }=-\epsilon \ddot{\text{ξ}}\cos \theta +u\end{array}\}$

(13.21)

where ξ is the displacement of the cart, u is the normalized input torque, w is the normalized spring force which serves as a disturbance and θ is the angular position of the proof body. The coupling between the translational and the rotational motions is governed by the parameter ε which is defined by

$\epsilon =\frac{me}{\sqrt{\left(I+m{e}^2\right)\left(M+m\right)}},0\le \epsilon \le 1,$

where e is the eccentricity of the pendulum, and ε = 0 if and only if e = 0. In this event, the dynamics reduces to

$\begin{array}{l}\ddot{\text{ξ}}+\text{ξ}=w\\ \ddot{\theta }=u\end{array}\}$

(13.22)

and is not stabilizable, since it is completely decoupled. However, if we let $x:={\left[\begin{array}{cccc}{x}_1& x_2& x_3& x_4\end{array}\right]}^T={\left[\begin{array}{cccc}\text{ξ}& \dot{\text{ξ}}& \theta & \dot{\theta }\end{array}\right]}^T,$ then the former can be represented in state-space as

$\dot{x}=\left[\begin{array}{l}{x}_2\\ \frac{-x+\epsilon x_4^2\sin x_3}{1-{\epsilon }^2{\cos }^2{x}_3}\\ x_4\\ \frac{\epsilon \cos x_3\left(x_1-\epsilon x_4^2\sin x_3\right)}{1-{\epsilon }^2{\cos }^2{x}_3}\end{array}\right]+\left[\begin{array}{l}0\\ \frac{1}{1-{\epsilon }^2{\cos }^2{x}_3}\\ 0\\ \frac{-\epsilon \cos x_3}{1-{\epsilon }^2{\cos }^2{x}_3}\end{array}\right]w+\left[\begin{array}{l}0\\ \frac{-\epsilon \cos x_3}{1-{\epsilon }^2{\cos }^2{x}_3}\\ 0\\ \frac{1}{1-{\epsilon }^2{\cos }^2{x}_3}\end{array}\right]u,$

(13.23)

$where1-{\epsilon }^2{\cos }^2{x}_3\ne 0forall{x}_{3}and\epsilon <1.$

FIGURE 13.1
Nonlinear Benchmark Control Problem; Originally adopted from Int. J. of Robust and Nonlinear Control, vol. 8, pp. 307-310 © 1998,“A benchmark problem for nonlinear control design,” by R. T. Bupp et al., with permission from Wiley Blackwell.

We first consider the design of a simple linear controller that locally asymptotically stabilizes the system with w = 0. Consider the linear control law

$u=-k_1\theta -k_2\dot{\theta },k_1>0,k_2>0$

(13.24)

and the Lyapunov-function candidate

$V\left(x\right)=\frac{1}{2}{x}^{T}P\left(\theta \right)x=\frac{1}{2}{\dot{\xi }}^2+\frac{1}{2}{\dot{\theta }}^2+\epsilon \dot{\xi }\dot{\theta }\cos \theta +\frac{1}{2}{\xi }^2+\frac{1}{2}{k}_1{\theta }^2$

where

$P\left(\theta \right)=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& \epsilon \cos \theta \\ 0& 0& k_1& 0\\ 0& \epsilon \cos \theta & 0& 1\end{array}\right].$

The eigenvalues of P(θ) are {1, k₁, 1±ε cos θ}, and since 0 ≤ ε < 1, P(θ) is positive-definite. Moreover, along trajectories of the the closed-loop system (13.23), (13.24),

$\dot{V}=-k_2{\dot{\theta }}^2\le 0.$

Hence, the closed-loop system is stable. In addition, using LaSalle’s invariance-principle, local asymptotic-stability can be concluded.

Design of Nonlinear Controller

Next, we apply the procedure developed above to design a nonlinear ${\mathcal{H}}_{\infty }$-controller by solving the HJIE. For this, we note the system specifications and constraints:

$\left|\xi \right|\le 1.282,\epsilon =0.2,\left|u\right|\le 1.411,$

and we choose

$C=diag\left\{1,\sqrt{0.1},\sqrt{0.1},\sqrt{0.1}\right\}.$

We first expand f(x) and G(x) in the HJIE (13.14) using the basis functions x^[1], x^[2], x^[3] ,… as

$f\left(x\right)\approx \left[\begin{array}{l}{x}_2\\ -\frac{25}{24}{x}_1+\frac{5}{24}{x}_4^2{x}_3+\frac{25}{576}{x}_1{x}_3^2+\dots \\ \frac{5}{24}{x}_1-\frac{1}{24}{x}_4^2{x}_3-\frac{65}{576}{x}_1{x}_3^2+\dots \end{array}\right]$

$G\left(x\right)\approx \left[\begin{array}{cc}\begin{array}{l}0\\ \frac{25}{24}-\frac{25}{576}{x}_3^2+\dots \\ 0\\ -\frac{5}{24}+\frac{65}{576}{x}_3^2+\dots \end{array}& \begin{array}{l}0\\ -\frac{5}{24}+\frac{65}{576}{x}_3^2+\dots \\ 0\\ \frac{25}{24}-\frac{25}{576}{x}_1{x}_3^2+\dots \end{array}\end{array}\right].$

Then the linearized plant about x = 0 is given by the following matrices

$\begin{array}{l}A=\frac{\partial f}{\partial x}\left(0\right)=\left[\begin{array}{cccc}0& 1& 0& 0\\ -\frac{25}{24}& 0& 0& 0\\ 0& 0& 0& 1\\ \frac{5}{24}& 0& 0& 0\end{array}\right],B_1=G_1\left(0\right)=\left[\begin{array}{l}0\\ \frac{25}{24}\\ 0\\ -\frac{5}{24}\end{array}\right],\\ B_2=G_2\left(0\right)=\left[\begin{array}{l}0\\ -\frac{5}{24}\\ 0\\ \frac{25}{24}\end{array}\right].\end{array}$

We can now solve the ARE (13.10) with H₁ = C. It can be shown that for all γ > γ^⋆ = 5.5, the Riccati equation has a positive-definite solution. Choosing γ = 6, we get the solution

$P=\left[\begin{array}{cccc}19.6283& -2.8308& -0.7380& -2.8876\\ -2.8308& 15.5492& 0.8439& 1.9915\\ -0.7380& 0.8439& 0.3330& 0.4967\\ -2.8876& 1.9915& 0.4967& 1.4415\end{array}\right]>0$

yielding

$A_{\star }=\left[\begin{array}{cccc}0& 1.0000& 0& 0\\ -1.6134& 0.2140& 0.0936& 0.2777\\ 0& 0& 0& 1.000\\ 2.7408& 1.1222& -0.3603& -1.1422\end{array}\right]$

with eigenvalues {−0.0415±i1.0156, −0.4227±i0.3684}. The linear feedbacks resulting from the above P are also given by

$v_{\star }^{\left[1\right]}\left(x\right)={\left[\begin{array}{cc}{l}_{\star }^{\left[1\right]}& k_{\star }^{\left[1\right]}\end{array}\right]}^T=\left[\begin{array}{l}-0.0652x_1+0.4384x_2+0.0215x_3+0.0493x_4\\ 2.4182x_1+1.1650x_2-0.3416x_3-1.0867x_4\end{array}\right].$

Higher-Order Terms

Next, we compute the higher-order terms in the expansion. Since A_⋆ is Hurwitz, its eigenvalues are nonresonant, and so the system of linear equations for the coefficient matrix in (13.20) has a unique solution and the series converges. Moreover, for this example,

$f^{\left[2k\right]}\left(x\right)=0,G^{\left[2k-1\right]}\left(x\right)=0,k=1,2,\dots$

Thus, $V^{[3]}(x)=0,and{v}_{\star }^{[2]}(x)=0.$ The first non-zero higher-order terms are

$\begin{array}{l}{v}_{\star }^{\left[3\right]}\left(x\right)=-R^{-1}\left[B^T{V}_x^{\left[4\right]T}\left(x\right)+G^{\left[2\right]T}\left(x\right)\right],\\ V_x^{\left[4\right]}\left(x\right)A_{\star }x=-V_x^{\left[2\right]}\left(x\right)\left[f^{\left[3\right]}\left(x\right)+G^{\left[2\right]}{v}_{\star }^{\left[1\right]}\left(x\right)\right].\end{array}$

The above system of equations can now be solved to yield

$\begin{array}{l}{V}^{\left[4\right]}=162.117x_1^4+91.1375x_2^4-0.9243x_4{x}_3^2{x}_1-0.4143x_4{x}_3^2{x}_2-\\ 0.1911x_4{x}_3^3+59.6069x_4^2{x}_1^2-66.0818x_4^2{x}_2{x}_1+42.1915x_4^2{x}_2^2-\\ 8.7947x_4^2{x}_3{x}_{1}2+0.6489x_4^4-151.8854x_4{x}_1^3+235.0386x_4{x}_2{x}_1^2-\\ 193.9184x_4{x}_2^2{x}_1+96.2534x_4{x}_2^3+43.523x_4{x}_3{x}_1^2-45.9440x_4{x}_3{x}_2{x}_1-\\ 258.0269x_2{x}_1^3+330.6741x_2^2{x}_1^2-186.0128x_3^2{x}_1-49.3835x_3{x}_1^3+\\ 92.8910x_3{x}_2{x}_1^2-70.7044x_3{x}_2^2{x}_1-0.22068x_3^3{x}_2+37.1118x_4{x}_3{x}_2^2+\\ 9.1538x_4^2{x}_3{x}_2-0.2863x_4^2{x}_3^2-9.6367x_4^3{x}_1+8.0508x_4^3{x}_2+\\ 0.7288x_4^3{x}_3+46.1405x_3{x}_2^3+9.4792x_3^2{x}_1^2-6.2154x_3^2{x}_2{x}_1+\\ 8.4205x_3^2{x}_2^2-0.4965x_3^3{x}_1-0.0156x_3^4\end{array}$

and

$\begin{array}{l}{v}_{\star }^{\left[3\right]}\left(x\right)\triangleq {\left[\begin{array}{cc}{l}_{\star }^{\left[3\right]}& k_{\star }^{\left[3\right]}\end{array}\right]}^T=\\ \left[\begin{array}{l}0.1261x_4^2{x}_3-0.0022x_4{x}_3^2-0.8724x_4^2{x}_1+1.1509x_4^2{x}_2+0.1089x_4^3+\\ 1.0208x_4{x}_3{x}_2+1.8952x_3{x}_2^2+0.2323x_3^2{x}_2+3.0554x_4{x}_1^2-5.2286x_4{x}_2{x}_1+\\ 3.9335x_4{x}_2^2-0.6138x_4{x}_3{x}_1-1.9128x_3{x}_2{x}_1-0.0018x_3^3+\\ 8.8880x_2{x}_1^2-7.5123x_2^2{x}_1+1.2179x_3{x}_1^2-3.2935x_1^3+4.9956x_2^3-0.0928x_1{x}_3^2\\ \\ -0.1853x_4^2{x}_3+0.0929x_4{x}_3^2+8.1739x_4^2{x}_1-3.7896x_4^2{x}_2-0.5132x_4^3-\\ 1.8036x_4{x}_3{x}_2-4.9101x_3{x}_2^2+0.3018x_3^2{x}_2-37.6101x_4{x}_1^2+28.4355x_4{x}_2{x}_1-\\ 13.8702x_4{x}_2^2+4.3753x_4{x}_3{x}_1+9.1991x_3{x}_2{x}_1+0.0043x_3^3-53.5255x_2{x}_1^2+\\ 42.8702x_2^2{x}_1-12.9923x_3{x}_1^2+52.2292x_1^3-12.1508x_2^3+0.02812x_1{x}_3^2\end{array}\right].\end{array}$

Remark 13.1.1 The above computational results give an approximate solution to the HJIE and the control law up to order three. Some difficulties that may be encountered include how to check the positive-definiteness of the solutions in general. Locally around x = 0 however, the positive-definiteness of P is reassuring, but does not guarantee global positive-definiteness and a subsequent global asymptotic-stabilizing controller. The computational burden of the algorithm also limits its attractiveness.

In the next section we discuss exact methods for solving the HJIE which may yield global solutions.