13.2  A Factorization Approach for Solving the HJIE

In this section, we discuss a factorization approach that may yield exact global solutions of the HJIE for the class of affine nonlinear systems. We begin with a discussion of sufficiency conditions for the existence of exact solutions to the HJIE (13.9) which are provided by the Implicit-function Theorem [157]. In this regard, let us write HJIE (13.9) in the form:

$HJI\left(x,V_x\right)=0,\text{}\text{}\text{}x\in M\subset \mathcal{X},$

(13.25)

where HJI : T ^⋆ M →ℜ. Then we have the following theorem.

Theorem 13.2.1 Assume that V ∈ C²(M), and the functions f(.), g₁(.), g₂(.), h(.) are smooth C²(M) functions. Then HJI(., .) is continuously-differentiable in an open neighborhood N × Ψ ⊂ T ^⋆ M of the origin. Furthermore, let $(\overline{x},{\overline{V}}_x)$ be a point in N × Ψ such that $HJI(\overline{x},{\overline{V}}_x)=0$ and the $ℱ$-derivative of HJI(., .) with respect to V_x is nonzero, i.e., $\frac{\partial }{\partial V_x}HJI(\overline{x},{\overline{V}}_x)\ne 0$ then there exists a continuously-differentiable solution:

$V_x\left(x\right)=\overline{HJI}\left(x\right)$

(13.26)

for some function $\overline{HJI}:N\to \Re ,ofHJIE$ (13.9) in N ×Ψ.

Proof: The proof of the above theorem follows from standard results of the Implicit-function Theorem [291]. This can also be shown by linearization of HJI around $(\overline{x},{\overline{V}}_x)$; the existence of such a point is guaranteed from the linear ${\mathcal{H}}_{\infty }$-control results [292]. Accordingly,

$HJI\left(x,V_x\right)=HJI\left(\overline{x},{\overline{V}}_x\right)+\left(V_x-{\overline{V}}_x\right)\frac{\partial HJI}{\partial V_x}\left(\overline{x},{\overline{V}}_x\right)+{\left(x-\overline{x}\right)}^T\frac{\partial HJI}{\partial x}\left(\overline{x},\overline{V_x}\right)+HOT=0$

(13.27)

where HOT denote higher-order terms. Since $HJI(\overline{x},{\overline{V}}_x)=0$, it follows from (13.27) that there exists a ball $B\left(\overline{x},{\overline{V}}_x;r\right)\in N\times \text{Ψ}$ of radius r > 0 centered at $(\overline{x},{\overline{V}}_x)$ such that in the limit as r → 0, HOT → 0 and V_x can be expressed in terms of x. □

Remark 13.2.1 Theorem 13.2.1 is only an existence result, and hence is not satisfactory, in the sense that it does not guarantee the uniqueness of V_x and it is only a local solution.

The objective of the approach then is to find an expression for V_x from the HJIE so that V can be recovered from it by carrying out the line-integral ${\int }_0^x{V}_x(\sigma )d\sigma .$ The integration is taken over any path joining the origin to x. For convenience, this is usually done along the axes as:

$\begin{array}{l}V\left(x\right)={\displaystyle {\int }_0^{x_1}{V}_{x_1}\left(y_1,0,\mathrm{\dots },0\right)d{y}_1}+{\displaystyle {\int }_0^{x_2}{V}_{x_2}\left(x_1,y_2,\mathrm{\dots },0\right)d{y}_2+\mathrm{\dots }+}\\ {\displaystyle {\int }_0^{x_n}{V}_{x_n}\left(x_1,x_2,\mathrm{\dots },y_n\right)d{y}_n.}\end{array}$

In addition, to ensure that V_x is the gradient of the scalar function V, it is necessary and sufficient that the Hessian matrix V_xx is symmetric for all x ∈ N. This will be referred to as the “curl-condition”:

$\frac{\partial V_{x_i}\left(x\right)}{\partial x_j}=\frac{\partial V_{x_j}\left(x\right)}{\partial x_i},i,j=1,\mathrm{\dots },n.$

(13.28)

Further, to account for the HOT in the Taylor-series expansion above, we introduce a parameter ζ = ζ(x) into the solution (13.26) as

$V_x\left(x\right)=\overline{HJI}\left(x,\zeta \right).$

The next step is to search for ξ ∈ T M such that the HJIE (13.9) is satisfied together with the curl conditions (13.28).

To proceed, let

${\mathcal{Q}}_{\gamma }\left(x\right)=\left[\frac{1}{{\gamma }^2}{g}_1\left(x\right)g_1^T\left(x\right)-g_2\left(x\right)g_2^T\left(x\right)\right],x\in M,$

then we have the following result.

Theorem 13.2.2 Consider the HJIE (13.9) and suppose there exists a vector field ζ : N → T N such that

${\zeta }^T\left(x\right){\mathcal{Q}}_{\gamma }^{+}\left(x\right)\zeta \left(x\right)-f^T\left(x\right){\mathcal{Q}}_{\gamma }^{+}\left(x\right)f\left(x\right)+h^T\left(x\right)h\left(x\right)\le 0;\forall x\in N,$

(13.29)

where the matrix ${\mathcal{Q}}_{\gamma }^{+}$ is the generalized-inverse of ${\mathcal{Q}}_{\gamma }$, then

$V_x=-\left(f\left(x\right)\right)\pm \zeta {\left(x\right)}^T{\mathcal{Q}}_{\gamma }^{+}\left(x\right),x\in N$

(13.30)

satisfies the HJIE (13.9).

Proof: By direct substitution and using the properties of generalized inverses [232]. □

Remark 13.2.2 ζ is referred to as the “discriminant-factor” of the system or the HJIE and (13.29) as the “discriminant-equation.” Moreover, since V_x(0) = 0, we require that ζ(0) = 0. This also holds for any equilibrium-point x_e of the system Σ.

Consider now the Hessian matrix of V from the above expression (13.30) which is given by:

$V_{xx}\left(x\right)=-{\left(\frac{\partial f\left(x\right)}{\partial x}\pm \frac{\partial \zeta \left(x\right)}{\partial x}\right)}^T{\mathcal{Q}}_{\gamma }^{+}\left(x\right)-\left(I_n\otimes {\left(f\left(x\right)\pm \zeta \left(x\right)\right)}^T\right)\frac{\partial {\mathcal{Q}}_{\gamma }^{+}\left(x\right)}{\partial x},$

(13.31)

where

$\begin{array}{l}\frac{\partial {\mathcal{Q}}_{\gamma }^{+}\left(x\right)}{\partial x}={\left[\frac{\partial {\mathcal{Q}}_{\gamma }^{+}\left(x\right)}{\partial x_1},\mathrm{\dots },\frac{\partial {\mathcal{Q}}_{\gamma }^{+}}{\partial x_n}\right]}^T,\frac{\partial f}{\partial x}=f_x\left(x\right)=\left[\frac{\partial f_1}{\partial x},\mathrm{\dots },\frac{\partial f_n}{\partial x}\right],\\ \frac{\partial \zeta }{\partial x}={\zeta }_x\left(x\right)=\left[\frac{\partial {\zeta }_1}{\partial x},\mathrm{\dots },\frac{\partial {\zeta }_n}{\partial x}\right],h_x\left(x\right)=\frac{\partial h}{\partial x}\left(x\right)=\left[\frac{\partial h_1}{\partial x},\mathrm{\dots },\frac{\partial h_m}{\partial x}\right].\end{array}$

Then, the Curl-conditions $V_{xx}(x)=V_{xx}^T(x)$ will imply

$\begin{array}{l}{\left(\frac{\partial f\left(x\right)}{\partial x}\pm \frac{\partial \zeta \left(x\right)}{\partial x}\right)}^T{\mathcal{Q}}_{\gamma }^{+}\left(x\right)+\left(I_n\otimes {\left(f\left(x\right)\pm \zeta \left(x\right)\right)}^T\right)\frac{\partial {\mathcal{Q}}_{\gamma }^{+}\left(x\right)}{\partial x}=\\ {\mathcal{Q}}_{\gamma }^{+}\left(x\right)\left(\frac{\partial f\left(x\right)}{\partial x}\pm \frac{\partial \zeta \left(x\right)}{\partial x}\right)+\frac{\partial {\mathcal{Q}}_{\gamma }^{+}\left(x\right)}{\partial x}\left(I_n\otimes \left(f\left(x\right)\pm \zeta \left(x\right)\right)\right)\end{array}$

(13.32)

which reduces to

${\left(\frac{\partial f\left(x\right)}{\partial x}\pm \frac{\partial \zeta \left(x\right)}{\partial x}\right)}^T{\mathcal{Q}}_{\gamma }^{+}={\mathcal{Q}}_{\gamma }^{+}\left(\frac{\partial f\left(x\right)}{\partial x}\pm \frac{\partial \zeta \left(x\right)}{\partial x}\right),x\in N$

(13.33)

if ${\mathcal{Q}}_{\gamma }^{+}\left(x\right)$ is a constant matrix.

Equations (13.32), (13.33) are a system of $\frac{n(n-1)}{2}$ first-order PDEs in n unknowns, ζ, which could be solved for ζ up to an arbitrary vector λ ∈ T N (cf. these conditions with the variable-gradient method for finding a Lyapunov-function [157]).

Next, the second requirement V_xx ≥ 0 will imply that

${\left(\frac{\partial f\left(x\right)}{\partial x}\pm \frac{\partial \zeta \left(x\right)}{\partial x}\right)}^T{\mathcal{Q}}_{\gamma }^{+}\left(x\right)+\left(I_n\otimes {\left(f\left(x\right)\pm \zeta \left(x\right)\right)}^T\right)\frac{\partial {\mathcal{Q}}_{\gamma }^{+}\left(x\right)}{\partial x}\le 0,\forall x\in N,$

(13.34)

and can only be imposed after a solution to (13.32) has been obtained.

Let us now specialize the above results to the a linear system

$\begin{array}{l}{\mathrm{\Sigma }}_l:\dot{x}=Fx+G_{1}w+G_{2}u\\ z\left(t\right)=\left[\begin{array}{l}{H}_{1}x\\ u\end{array}\right],\end{array}$

where $F\in {\Re }^{n\times n},G_1\in {\Re }^{n\times r},\text{and}{\text{G}}_2\in {\Re }^{n\times k},H_1\in {\Re }^{m\times n}$ are constant matrices. Then, in this case, $\zeta \left(x\right)=\Gamma x,\Gamma \in {\Re }^{n\times n},Q_{\gamma }=\left(\frac{1}{{\gamma }^2}{G}_1{G}_1^T-G_2{G}_2^T\right).$ Thus, V_x is given by

$V_x\left(x\right)=-{\left(F_x\pm {\text{Γ}}_x\right)}^T{\mathcal{Q}}_{\gamma }^{+}=-x^T{\left(F\pm \text{Γ}\right)}^T{\mathcal{Q}}_{\gamma }^{+}$

(13.35)

where Γ satisfies:

${\text{Γ}}^T{\mathcal{Q}}_{\gamma }^{+}\text{Γ}-F^T{\mathcal{Q}}_{\gamma }^{+}F+H^{T}H=0.$

Now define

$\text{Δ}=F^T{\mathcal{Q}}_{\gamma }^{+}F-H_1^T{H}_1,$

then Γ is given by

${\text{Γ}}^T{\mathcal{Q}}_{\gamma }^{+}\text{Γ}=\text{Δ}.$

(13.36)

This suggests that Γ is a coordinate-transformation matrix, which in this case is called a congruence transformation between ${\mathcal{Q}}_{\gamma }^{+}$ and Δ. Moreover, from (13.35), V_xx is given by

$V_{xx}=-{\left(F\pm \text{Γ}\right)}^T{\mathcal{Q}}_{\gamma }^{+}$

(13.37)

(cf. with the solution of the linear ${\mathcal{H}}_{\infty }$-Riccati equation $P=X_2{X}_1^{-1},\text{where}\text{the}\text{columns}\text{of}\left[\begin{array}{c}{X}_1\\ X_2\end{array}\right]\text{span}{\text{Λ}}_{H\gamma }^{-},$ the stable-eigenspace of the Hamiltonian-matrix associated with the linear ${\mathcal{H}}_{\infty }$ problem, with X₁ invertible [68, 292]). A direct connection of the above results with the Hamiltonian matrices approach for solving the ARE can be drawn from the following fact (see also Theorem 13.2, page 329 in [292]): If X ∈ C^n×n is a solution of the ARE

$A^{T}X+XA+XRX+Q=0,$

(13.38)

then there exist matrices X₁, X₂ ∈ $\mathcal{C}$^n×n, with X₁ invertible, such that $X=X_2{X}_1^{-1}$ and the columns of $\left[\begin{array}{c}{X}_1\\ X_2\end{array}\right]$ form a basis for an n-dimensional invariant-subspace of H defined by:

$H\triangleq \left[\begin{array}{cc}A& R\\ -Q& -A^T\end{array}\right].$

In view of the above fact, we now prove that the columns of $\left[\begin{array}{l}I\\ P\end{array}\right]\triangleq \left[\begin{array}{l}I\\ -{\left(F\pm \text{Γ}\right)}^T{\mathcal{Q}}_{\gamma }^{+}\end{array}\right]$ span an n-dimensional invariant-subspace of the Hamiltonian matix

$H_{\gamma }^l=\left[\begin{array}{cc}F& \left(\frac{1}{{\gamma }^2}{G}_1{G}_1^T-G_2{G}_2^T\right)\\ -H^{T}H& -F^T\end{array}\right]\triangleq \left[\begin{array}{cc}F& Q_{\gamma }\\ -H^{T}H& -F^T\end{array}\right]$

(13.39)

corresponding to the ARE (13.38).

Theorem 13.2.3 Suppose there exists a Γ that satisfies (13.36) and such that $P=-{\left(F\pm \text{Γ}\right)}^T{Q}_{\gamma }^{+}$ is symmetric, then P is a solution of ARE (13.10). Moreover, if Q_γ is nonsingular, then the columns of $\left[\begin{array}{c}I\\ P\end{array}\right]$ span an n-dimensional invariant subspace of $H_{\gamma }^l$. Otherwise, of $\left[\begin{array}{cc}{\mathcal{Q}}_{\gamma }^{+}& 0\\ 0& I\end{array}\right]H_{\gamma }^l.$

Proof: The first part of the theorem has already been shown. For the second part, using the symmetry of P we have

$\begin{array}{l}\left[\begin{array}{cc}{\mathcal{Q}}_{\gamma }^{+}& 0\\ 0& I\end{array}\right]\left[\begin{array}{cc}F& Q_{\gamma }\\ -H^{T}H& -F^T\end{array}\right]\left[\begin{array}{l}I\\ -{\left(F\pm \text{Γ}\right)}^T{\mathcal{Q}}_{\gamma }^{+}\end{array}\right]\\ =\left[\begin{array}{cc}{\mathcal{Q}}_{\gamma }^{+}& 0\\ 0& I\end{array}\right]\left[\begin{array}{cc}F& Q_{\gamma }\\ -H^{T}H& -F^T\end{array}\right]\left[\begin{array}{l}I\\ {\mathcal{Q}}_{\gamma }^{+}+\left(F\pm \text{Γ}\right)\end{array}\right]\\ =\left[\begin{array}{cc}{\mathcal{Q}}_{\gamma }^{+}& 0\\ 0& I\end{array}\right]\left[\begin{array}{l}I\\ -{\left(F\pm \text{Γ}\right)}^T{\mathcal{Q}}_{\gamma }^{+}\end{array}\right]\left(\mp \text{Γ}\right).\end{array}$

(13.40)

Hence, P defined as above indeed spans an n-dimensional invariant-subspace of $H_{\gamma }^l\text{if}{\mathcal{Q}}_{\gamma }$ is nonsingular. □

13.2.1  Worked Examples

In this subsection, we consider worked examples solved using the approach outlined in the previous section.

Example 13.2.1 Consider the following system:

$\begin{array}{l}{\dot{x}}_1=-x_1^2\left(t\right)-x_2\left(t\right)\\ {\dot{x}}_2=x_2+u+w\\ z=\left[\begin{array}{l}{x}_2\\ u\end{array}\right].\end{array}$

Let γ = 2, then

$\begin{array}{l}f\left(x\right)=\left[\begin{array}{l}-x_1^3+x_2\\ x_2\end{array}\right];G_1=\left[\begin{array}{l}0\\ 1\end{array}\right];G_2=\left[\begin{array}{l}0\\ 1\end{array}\right];h\left(x\right)=\left[\begin{array}{l}0\\ x_2\end{array}\right],\\ {\mathcal{Q}}_{\gamma }=\left[\begin{array}{cc}0& 0\\ 0& -\frac{3}{4}\end{array}\right],Q_{\gamma }^{+}=\left[\begin{array}{cc}0& 0\\ 0& -\frac{4}{3}\end{array}\right].\end{array}$

Substituting the above functions in (13.29), (13.33), (13.34), we get

$-4{\zeta }_2^2+4x_2^2+3x_2^2\le 0;\forall x\in N_e$

(13.41)

${\zeta }_{1,x_2}\left(x\right)=-1\forall x\in N_e$

(13.42)

$\left[\begin{array}{cc}0& \frac{4}{3}-\frac{4}{3}{\zeta }_{1,x_2}\\ 0& -\frac{4}{3}-\frac{4}{3}{\zeta }_{1,x_2}\end{array}\right]\le 0;\forall x\in N_e.$

(13.43)

Solving the above system we get

${\zeta }_2\left(x\right)=\pm \frac{\sqrt{7}}{2}{x}_2,{\zeta }_1\left(x\right)=-x_2+\varphi \left(x_1\right)$

where $\varphi \left(x_1\right)$ is some arbitrary function which we can take as Φ(x) = 0 without any loss of generality. Thus,

$V_x\left(x\right)=-{\left(f\left(x\right)\pm \zeta \left(x\right)\right)}^T{\mathcal{Q}}^{+}=\left(\begin{array}{cc}0& \frac{8\pm 4\sqrt{7}}{6}{x}_2\end{array}\right).$

Finally, integrating the positive term in the expression for V_x above from 0 to x, we get

$V\left(x\right)=\frac{2+\sqrt{7}}{3}{x}_2^2$

which is positive-semidefinite.

The next example will illustrate a general transformation approach for handling the discriminant equation/inequality and symmetry condition.

Example 13.2.2 Consider the following example with the disturbance affecting the first state equation and a weighting on the states:

$\begin{array}{l}{\dot{x}}_1=-x_1^3\left(t\right)-x_2\left(t\right)+w_1\\ {\dot{x}}_2=x_1+x_2+u+w_2\\ z=\left[\begin{array}{l}{Q}^{\frac{1}{2}}x\\ R^{\frac{1}{2}}u\end{array}\right]\end{array}$

where Q = diag{q₁, q₂} ≥ 0, R = r > 0 are weighting matrices introduced to make the HJIE solvable. The state-feedback HJIE associated with the above system is thus given by

$V_x\left(x\right)f\left(x\right)+\frac{1}{2}{V}_x\left(x\right)\left[\frac{1}{{\gamma }^2}{g}_1\left(x\right)g_1^T\left(x\right)-g_2\left(x\right)R^{-1}{g}_2^T\left(x\right)\right]V_x^T\left(x\right)+\frac{1}{2}{x}^T{Q}^x\le 0,$

(13.44)

with $V\left(0\right)=0,{\mathcal{Q}}_{\gamma }\left(x\right)=\left[\frac{1}{{\gamma }^2}{g}_1\left(x\right)g_1^T\left(x\right)-g_2\left(x\right)R^{-1}{g}_2^T\left(x\right)\right].$ Then

$f\left(x\right)=\left[\begin{array}{l}-x_1^3-x_2\\ x_1+x_2\end{array}\right],G_2\left[\begin{array}{l}0\\ 1\end{array}\right],G_1=I_2,{\mathcal{Q}}_{\gamma }=\left[\begin{array}{cc}\frac{1}{{\gamma }^2}& 0\\ 0& \frac{{\text{γ-γ}}^2}{{\text{γγ}}^2}\end{array}\right],{\mathcal{Q}}_{\gamma }^{+}=\left[\begin{array}{cc}\frac{1}{{\gamma }^2}& 0\\ 0& \frac{{\text{γγ}}^2}{{\text{γ-γ}}^2}\end{array}\right].$

Substituting the above functions in (13.29), (13.33), (13.34), we get

${\gamma }^2{\zeta }_1^2+\frac{r{\gamma }^2}{r-{\gamma }^2}{\zeta }_2^2-{\gamma }^2{\left(x_1^3+x_2\right)}^2-\frac{r{\gamma }^2}{r-{\gamma }^2}{\left(x_1+x_2\right)}^2+q_1{x}_1^2+q_2{x}_2^2=0$

(13.45)

$\frac{r{\gamma }^2}{r-{\gamma }^2}{\zeta }_{2,x_1}-{\zeta }_{1,x_2}=\frac{{\gamma }^2}{r-{\gamma }^2}$

(13.46)

$\left[\begin{array}{cc}{\gamma }^2\left({\zeta }_{1,x_1}-3x_1^2\right)& \frac{r{\gamma }^2}{r-{\gamma }^2}\left({\zeta }_{2x_1}+1\right)\\ {\gamma }^2\left({\zeta }_{1,x_2}-1\right)& \frac{r{\gamma }^2}{r-{\gamma }^2}\left({\zeta }_{2,x_2}+1\right)\end{array}\right]\le 0.$

(13.47)

One way to handle the above algebraic-differential system is to parameterize ζ₁ and ζ₂ as:

${\zeta }_1\left(x\right)=a{x}_1+b{x}_2,{\zeta }_2\left(x\right)=c{x}_1+d{x}_2+e{x}_1^3,$

where a, b, c, d, e ∈ ℜ are constants, and to try to solve for these constants. This approach may not however work for most systems. We therefore illustrate next a more general procedure for handling the above system.

Suppose we choose r, γ such that $\frac{r{\gamma }^2}{r-{\gamma }^2}>0$

(usually we take r >> 1 and 0 < γ < 1).We now apply the following transformation to separate the variables:

${\zeta }_1\left(x\right)=\frac{1}{\gamma }\rho \left(x\right)\cos \theta \left(x\right),{\zeta }_2\left(x\right)\sqrt{\frac{\left(r-{\gamma }^2\right)}{r{\gamma }^2}}\rho \left(x\right)\sin \theta \left(x\right).$

where ρ, θ : N → ℜ are C² functions. Substituting in the equation (13.45), we get

$\rho \left(x\right)=\pm \sqrt{\frac{r{\gamma }^2}{r-{\gamma }^2}{\left(x_1+x_2\right)}^2-{\gamma }^2{\left(x_1^3+x_2\right)}^2-q_1{x}_1^2-q_2{x}_2^2}.$

Thus, for the HJIE (13.44) to be solvable, it is necessary that γ, r, q₁, q₂ are chosen such that the function under the square-root in the above equation is positive for all x ∈ N so that ρ is real. As a matter of fact, the above expression defines N, i.e., N = {x | ρ ∈ ℜ}.

If however we choose r, γ such that $\frac{r{\gamma }^2}{r-{\gamma }^2}<0,$ then we must paramerize ${\zeta }_1,{\zeta }_{2}as{\zeta }_1\left(x\right)=\frac{1}{\gamma }\rho \left(x\right)\cosh \theta \left(x\right),{\zeta }_2\left(x\right)=\sqrt{\frac{\left(r-{\gamma }^2\right)}{r{\gamma }^2}}\rho \left(x\right)\sinh \theta \left(x\right).$ A difficulty also arises when ${\mathcal{Q}}_{\gamma }^{+}\left(x\right)$ is not diagonal, e.g., if ${\mathcal{Q}}_{\gamma }^{+}=\left[\begin{array}{cc}a& b\\ 0& c\end{array}\right],a,b,c\in \Re .Then{\zeta }^T\left(x\right){\mathcal{Q}}_{\gamma }^{+}\zeta \left(x\right)=i{\zeta }_1^2\left(x\right)+j{\zeta }_1\left(x\right){\zeta }_2\left(x\right)+k{\zeta }_2^2\left(x\right)$, for some i, j, k ∈ ℜ. The difficulty here is created by the cross-term jζ₁ζ₂ as the above parameterization cannot lead to a simplification of the problem. However, we can use a completion of squares to get

${\zeta }^T\left(x\right){\mathcal{Q}}_{\gamma }^{+}\zeta \left(x\right)={\left(\sqrt{i}{\zeta }_1\left(x\right)+\frac{i}{2\sqrt{i}}{\zeta }_2\left(x\right)\right)}^2+\left(k-\frac{j^2}{4i}\right){\zeta }_2^2\left(x\right)$

(assuming i > 0, otherwise, pull out the negative sign outside the bracket). Now we can define

$\left(\sqrt{i}{\zeta }_1\left(x\right)+\frac{1}{2\sqrt{i}}{\zeta }_2\left(x\right)\right)=\rho \left(x\right)\cos \theta \left(x\right),and{\zeta }_2\left(x\right)={\left(k-\frac{j^2}{4i}\right)}^{-1/2}\rho \left(x\right)\sin \theta \left(x\right).$ Thus, in reality,

${\zeta }_1\left(x\right)=\rho \left(x\right)\left(\frac{1}{\sqrt{i}}\cos \theta \left(x\right)-\frac{j}{2i}{\left(k-\frac{j^2}{4i}\right)}^{1/2}\sin \theta \left(x\right)\right).$

Next, we determine θ(.) from (13.46). Differentiating ζ₁, ζ₂ with respect to x₂ and x₁ respectively and substituting in (13.46), we get

$\begin{array}{l}\beta \left({\rho }_{x_1}\left(x\right)\sin \theta \left(x\right)+\rho \left(x\right)\theta x_1\left(x\right)\cos \theta \left(x\right)\right)-\\ \kappa \left({\rho }_{x_2}\left(x\right)\cos \theta \left(x\right)-\rho \left(x\right){\theta }_{x_2}\left(x\right)\sin \theta \left(x\right)\right)=\eta \end{array}$

(13.48)

where $\beta =\frac{1}{\gamma }\sqrt{\frac{r}{\left(r-{\gamma }^2\right)}},\kappa =\frac{1}{\gamma },\eta =\frac{{\gamma }^2}{r-{\gamma }^2}.$ This first-order PDE in θ can be solved using the method of “characteristics” discussed in Chapter 4 (see also the reference [95]). However, the geometry of the problem calls for a simpler approach. Moreover, since θ is a free parameter that we have to assign to guarantee that V_xx is symmetric and positive-(semi) definite, there are many solutions to the above PDE. One solution can be obtained as follows. Rearranging the above equation, we get

$\left(\beta {\rho }_{x_1}\left(x\right)+\kappa \rho \left(x\right){\theta }_{x_2}\left(x\right)\right)\sin \theta \left(x\right)+\left(-\kappa {\rho }_{x_2}\left(x\right)+\beta \rho \left(x\right){\theta }_{x_1}\left(x\right)\right)\cos \theta \left(x\right)=\eta .$

(13.49)

If we now assign

$\begin{array}{l}\left(\beta {\rho }_{x_1}\left(x\right)+\kappa \rho \left(x\right){\theta }_{x_2}\left(x\right)\right)=\frac{\eta }{2}\sin \theta \left(x\right),\\ \left(-\kappa {\rho }_{x_2}\left(x\right)+\beta \rho \left(x\right){\theta }_{x_1}\left(x\right)\right)=\frac{\eta }{2}\cos \theta \left(x\right),\end{array}$

then we see that (13.49) is satisfied. Further, squaring both sides of the above equations and adding, we get

$\frac{4}{{\eta }^2}{\left(\beta {\rho }_{x_1}\left(x\right)+\kappa \rho \left(x\right){\theta }_{x_2}\left(x\right)\right)}^2+\frac{4}{{\eta }^2}{\left(-\kappa {\rho }_{x_2}\left(x\right)+\beta \rho \left(x\right){\theta }_{x_1}\left(x\right)\right)}^2=1$

(13.50)

which is the equation of an ellipse in the coordinates θ_x1, θ_x2, centered at $\frac{k\rho x_2\left(x\right)}{\beta \rho \left(x\right)},-\frac{k\rho x_1\left(x\right)}{\beta \rho \left(x\right)}andradii\frac{\eta }{2\beta \rho \left(x\right)},\frac{\eta }{2k\rho \left(x\right)}$ respectively. Thus, any point on this ellipse will give the required gradient for θ. One point on this ellipse corresponds to the following gradients in θ:

$\begin{array}{l}{\theta }_{x_1}\left(x\right)=\frac{\kappa {\rho }_{x_2}\left(x\right)}{\beta \rho \left(x\right)}+\frac{1}{\sqrt{2}}\left(\frac{\eta }{2\beta \rho \left(x\right)}\right)\\ {\theta }_{x_2}\left(x\right)=-\frac{\beta {\rho }_{x_1}\left(x\right)}{\kappa \rho \left(x\right)}+\frac{1}{\sqrt{2}}\left(\frac{\eta }{2\kappa \rho \left(x\right)})\right).\end{array}$

Hence, we can finally obtain θ as

$\theta \left(x\right)={{\displaystyle {\int }_{0+}^{x_1}\left(\frac{\kappa {\rho }_{x_2}\left(x\right)}{\beta \rho \left(x\right)}+\frac{1}{\sqrt{2}}\left(\frac{\eta }{2\beta \rho \left(x\right)}\right)\right)|}}_{x_2=0}d{x}_1+{\displaystyle {\int }_{0+}^{x_2}\left(-\frac{\beta {\rho }_{x_1}\left(x\right)}{\kappa \rho \left(x\right)}+\frac{1}{\sqrt{2}}\left(\frac{\eta }{2\kappa \rho \left(x\right)})\right)\right)}d{x}_2$

The above integral can be evaluated using Mathematica or Maple. The result is very complicated and lengthy, so we choose not to report it here.

Remark 13.2.3 Note, any available method can also be used to solve the symmetry PDE in θ (13.48), as the above approach may be too restricted and might not yield the desired solution. Indeed, a general solution would be more desirable. Moreover, any solution should be checked against the positive-(semi)definite condition (13.47) to see that it is satisfied. Otherwise, some of the design parameters, r, γ, q_i, should be adjusted to see that this condition is at least satisfied.

Finally, we can compute V as

$V\left(x\right)=-{\displaystyle {\int }_{0+}^x{\left(f\left(x\right)+\rho \left(x\right)\left[\begin{array}{l}\cos \theta \left(x\right)\\ \sin \theta \left(x\right)\end{array}\right]\right)}^T{\mathcal{Q}}_{\gamma }^{+}dx.}$

(13.51)

Remark 13.2.4 It may not be necessary to compute V explicitly, since the optimal control u^⋆ = α(x) is only a function of V_x. What is more important is to check that the positive -(semi) definiteness condition (13.47) is locally satisfied around the origin {0}. Then the V function corresponding to this V_x will be a candidate solution for the HJIE. However, we still cannot conclude at this point that it is a stabilizing solution. In the case of the above example, it can be seen that by setting ζ₁(x) = ρ(x) cos θ(x), ζ₂(x) = ρ(x) sin θ(x), and their derivatives equal to 0, the inequality (13.47) is locally satisfied at the origin {0}.

Example 13.2.3 In this example, we consider the model of a satellite considered in [46, 87, 154, 169] (and the references there in). The equations of motion of the spinning satellite are governed by two subsystems; namely, a kinematic model and a dynamic model. The configuration space of the satellite is a six dimensional manifold the tangent bundle of SO(3), or TSO(3), (where SO(3) is the special orthogonal linear matrix group). The equations of motion are given by:

$\dot{R}=RS\left(\omega \right)$

(13.52)

$J\dot{\omega }=S\left(\omega \right)J\omega +u+Pd,$

(13.53)

where ω ∈ℜ³ is the angular velocity vector about a fixed inertial reference frame with three principal axes and having the origin at the center of gravity of the satellite, R ∈ SO(3), is the orientation matrix of the satellite, u ∈ℜ³ is the control torque input vector, and d is the vector of external disturbances on the spacecraft. P = diag{P₁, P₂, P₃}, P_i ∈ ℜ, i = 1, 2, 3, is a constant gain matrix, J is the inertia matrix of the system, and S(ω) is the skew-symmetric matrix

$S\left(\omega \right)=\left[\begin{array}{ccc}0& -{\omega }_3& {\omega }_2\\ {\omega }_3& 0& -{\omega }_1\\ -{\omega }_2& {\omega }_1& 0\end{array}\right].$

We consider the control of the angular velocities governed by the dynamic subsystem (13.53). By letting J = diag{I₁, I₂, I₃}, where I₁>0, I₂ > 0, I₃ > 0 and without any loss of generality assuming I₁ ≠ I₂ ≠ I₃ are the principal moments of inertia, the subsystem (13.53) can be represented as:

$\begin{array}{l}{I}_1{\dot{\omega }}_1\left(t\right)=\left(I_2-I_3\right){\omega }_2\left(t\right){\omega }_3\left(t\right)+u_1+P_1{d}_1\left(t\right)\\ I_2{\dot{\omega }}_2\left(t\right)=\left(I_3-I_1\right){\omega }_3\left(t\right){\omega }_1\left(t\right)+u_2+P_2{d}_2\left(t\right)\\ I_3{\dot{\omega }}_3\left(t\right)=\left(I_1-I_2\right){\omega }_1\left(t\right){\omega }_2\left(t\right)+u_3+P_3{d}_3\left(t\right).\end{array}$

Now define

$A_1=\frac{\left(I_2-I_3\right)}{I_1},A_2=\frac{\left(I_3-I_1\right)}{I_2},A_3=\frac{\left(I_1-I_2\right)}{I_3},$

then the above equations can be represented as:

$\begin{array}{l}\dot{\omega }\left(t\right):=f\left(\omega \right)+B_{1}d\left(t\right)+B_{2}v\left(t\right)\\ =\left[\begin{array}{l}{A}_1{\omega }_2{\omega }_3\\ A_2{\omega }_1{\omega }_3\\ A_3{\omega }_1{\omega }_2\end{array}\right]+\left[\begin{array}{ccc}{b}_1& 0& 0\\ 0& b_2& 0\\ 0& 0& b_3\end{array}\right]\left[\begin{array}{l}{d}_1\\ d_2\\ d_3\end{array}\right]+\\ \left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{l}{u}_1\\ u_2\\ u_3\end{array}\right],\end{array}$

(13.54)

where

$b_1=\frac{P_1}{I_1},b_2=\frac{P_2}{I_2},b_3=\frac{P_3}{I_3}.$

In this regard, consider the output function:

$z=h\left(\omega \right)=\left[\begin{array}{l}{c}_1{\omega }_1\\ c_2{\omega }_2\\ c_3{\omega }_3\end{array}\right]$

(13.55)

where c₁, c₂, c₃ are design parameters. Applying the results of Section 13.2, we have

$\begin{array}{l}{Q}_{\gamma }=\left(\frac{1}{{\gamma }^2}{B}_1{B}_1^T-B_2{B}_2^T\right)=\left[\begin{array}{ccc}\frac{b_1^2}{{\gamma }^2}-1& 0& 0\\ 0& \frac{b_2^2}{{\gamma }^2}-1& 0\\ 0& 0& \frac{b_3^2}{{\gamma }^2}-1\end{array}\right],\\ V_{\omega }\left(\omega \right)=-{\left(f\left(\omega \right)\pm \zeta \left(\omega \right)\right)}^T{Q}_{\gamma }^{+}\end{array}$

(13.56)

and (13.29) implies

${\zeta }^T\left(\omega \right)Q_{\gamma }^{+}\zeta \left(\omega \right)-f^T\left(\omega \right)Q_{\gamma }^{+}f\left(\omega \right)+h^T\left(\omega \right)h\left(\omega \right)\le 0.$

Upon substitution, we get

$\begin{array}{l}\left(\frac{{\gamma }^2}{b_1^2-{\gamma }^2}\right){\zeta }_1^2\left(\omega \right)+\left(\frac{{\gamma }^2}{b_2^2-{\gamma }^2}\right){\zeta }_2^2\left(\omega \right)+\left(\frac{{\gamma }^2}{b_3^2-{\gamma }^2}\right){\zeta }_3^2\left(\omega \right)-\left(\frac{{\gamma }^2}{b_1^2-{\gamma }^2}\right)A_1^2{\omega }_2^2{\omega }_3^2-\\ \left(\frac{{\gamma }^2}{b_2^2-{\gamma }^2}\right)A_2^2{\omega }_1^2{\omega }_3^2-\left(\frac{{\gamma }^2}{b_3^2-{\gamma }^2}\right)A_3^2{\omega }_1^2{\omega }_2^2+c_1^2{\omega }_1^2+c_2^2{\omega }_2^2+c_3^2{\omega }_3^2\le 0.\end{array}$

Further, substituting in (13.33), we have the following additional constraints on ζ:

$A_1{\omega }_3+{\zeta }_{1,{\omega }_2}={\zeta }_{2,{\omega }_1}+A_2{\omega }_3,A_1{\omega }_2+{\zeta }_{1,{\omega }_3}=A_3{\omega }_2+{\zeta }_{3,{\omega }_1},A_2{\omega }_1+{\zeta }_{3,{\omega }_1}=A_{3{\omega }_1}+{\zeta }_{1,{\omega }_3}.$

(13.57)

Thus, for any γ > b = max_i{b_i}, i = 1, 2, 3, and $c_i=\sqrt{\left|\left(\frac{{\gamma }^2}{b_i^2-{\gamma }^2}\right)\right|},i=1,2,3,$ and under the assumption A₁ + A₂ + A₃ = 0 (see also [46]), we have the following solution

$\begin{array}{l}{\zeta }_1=-A_1{\omega }_2{\omega }_3+{\omega }_1\\ {\zeta }_2=-A_2{\omega }_1{\omega }_3+{\omega }_2\\ {\zeta }_3=-A_3{\omega }_1{\omega }_2+{\omega }_3.\end{array}$

Thus,

$V_{\omega }\left(\omega \right)=-{\left(f\left(\omega \right)+\zeta \left(\omega \right)\right)}^T{Q}_{\gamma }^{+}=\left[\left(\frac{{\gamma }^2}{{\gamma }^2-b_1^2}\right){\omega }_1\left(\frac{{\gamma }^2}{{\gamma }^2-b_2^2}\right){\omega }_2\left(\frac{{\gamma }^2}{{\gamma }^2-b_3^2}\right){\omega }_3\right]$

and integrating from 0 to ω, we get

$V\left(\omega \right)=\frac{1}{2}\left(\left(\frac{{\gamma }^2}{{\gamma }^2-b_1^2}\right){\omega }_1^2+\left(\frac{{\gamma }^2}{{\gamma }^2-b_2^2}\right){\omega }_2^2+\left(\frac{{\gamma }^2}{{\gamma }^2-b_3^2}\right){\omega }_3^2\right)$

which is positive-definite for any γ > b.

Remark 13.2.5 Notice that the solution of the discriminant inequality does not only give us a stabilizing feedback, but also the linearizing feedback control. However, the linearizing terms drop out in the final expression for V, and consequently in the expression for the optimal control $u^{\star }=-B_2^T{V}_{\omega }^T\left(\omega \right).$ This clearly shows that cancellation of the nonlinearities is not optimal.

13.3  Solving the Hamilton-Jacobi Equation for Mechanical Systems and Application to the Toda Lattice

In this section, we extend the factorization approach discussed in the previous section to a class of Hamiltonian mechanical systems, and then apply the approach to solve the Toda lattice equations discussed in Chapter 4. Moreover, in Chapter 4 we have reviewed Hamilton’s transformation approach for integrating the equations of motion by introducing a canonical transformation which can be generated by a generating function also known as Hamilton’s principle function. This led to the Hamilton-Jacobi PDE which must be solved to obtain the required transformation generating function. However, as has been discussed in the previous sections, the HJE is very difficult to solve, except for the case when the Hamiltonian function is such that the equation is separable. It is therefore our objective in this section to present a method for solving the HJE for a class of Hamiltonian systems that may not admit a separation of variables.

13.3.1  Solving the Hamilton-Jacobi Equation

In this subsection, we propose an approach for solving the Hamilton-Jacobi equation for a fairly large class of Hamiltonian systems, and then apply the appproach to the ${\mathcal{A}}_2$ Toda-lattice as a special case. To present the approach, let the configuration space of the class of Hamiltonian systems be a smooth n-dimensional manifold M with local coordinates q = (q₁,…, q_n), i.e., if (φ, U) is a coordinate chart, we write φ = q and ${\dot{q}}_i=\frac{\partial }{\partial q_i}$ in the tangent bundle T M|_U = T U. Further, let the class of systems under consideration be represented by Hamiltonian functions H : T ^⋆ M → ℜ of the form:

$H\left(q,p\right)=\frac{1}{2}{\displaystyle \sum _{i=1}^n{p}_i^2+V\left(q\right)},$

(13.58)

where $\left(p_1\left(q\right),\dots p_n\left(q\right)\right)\in T_q^{\star }M,$ and together with (q₁,…, q_n) form the 2n symplectic-coordinates for the phase-space T ^⋆ M of any system in the class, while V : M →ℜ₊ is the potential function which we assume to be nonseparable in the variables q_i, i = 1,…,n. The time-independent HJE corresponding to the above Hamiltonian function is given by

$\frac{1}{2}{\displaystyle \sum _{i=1}^n{\left(\frac{\partial W}{\partial q_i}\right)}^2+V\left(q\right)=h,}$

(13.59)

where W : M →ℜ is the Hamilton’s characteristic-function for the system, and h is the energy constant. We then have the following main theorem for solving the HJE.

Theorem 13.3.1 Let M be an open subset of ℜⁿ which is simply-connected² and let q = (q₁,…, q_n) be the coordinates on M. Suppose ρ, θ_i : M → ℜ for $i=1,\dots ,\left[\frac{n+1}{2}\right];\theta =\left({\theta }_1,\dots ,{\theta }_{\left[\frac{n+1}{2}\right]}\right);and{\zeta }_i:\Re \times {\Re }^{\left[\frac{n+1}{2}\right]}\to \Re$

are C² functions such that

$\frac{\partial {\zeta }_i}{\partial q_j}\left(\rho \left(q\right),\theta \left(q\right)\right)=\frac{\partial {\zeta }_j}{\partial q_i}\left(\rho \left(q\right),\theta \left(q\right)\right),\forall i,j=1,\mathrm{\dots },n,$

(13.60)

and

$\frac{1}{2}{\displaystyle \sum _{i=1}^n{\zeta }_i^2\left(\rho \left(q\right),\theta \left(q\right)\right)+V\left(q\right)=h}$

(13.61)

is solvable for the functions ρ, θ. Let

${\omega }^1={\displaystyle \sum _{i=1}^n{\zeta }_i\left(\rho \left(q\right),\theta \left(q\right)\right)d{q}_i,}$

and suppose C is a path in M from an initial point q₀ to an arbitrary point q ∈ M. Then

(i)  ω¹ is closed;

(ii)  ω¹ is exact;

(iii)  if W (q) = ∫_C ω¹,then W satisfies the HJE (13.59).

Proof: (i)

$d{\omega }^1={\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^n\frac{\partial }{\partial q_j}{\zeta }_i\left(\rho \left(q\right),\theta \left(q\right)\right)}}d{q}_j\wedge d{q}_i,$

which by (13.60) implies dω¹ = 0. Hence, ω¹ is closed.³ (ii) Since by (i) ω¹ is closed, then by the simple-connectedness of M, ω¹ is also exact. (iii) By (ii) ω¹ is exact. Therefore, the integral $W\left(q\right)={\displaystyle {\int }_C{\omega }^1}$ is independent of the path C, and W corresponds to a scalar function. Furthermore, dW = ω¹ and $\frac{\partial W}{\partial q_i}={\zeta }_i\left(\rho \left(q\right),\theta \left(q\right)\right).$ Thus, substituting this in the HJE (13.59) and if (13.61) holds, then W satisfies it. □

In the next corollary we shall construct explicitly the functions ζ_i, i = 1,…,n in the above theorem.

Corollary 13.3.1 Assume the dimension n of the system is 2, and M, ρ, θ are as in the hypotheses of Theorem 13.3.1, and that conditions (13.60), (13.61) are solvable for θ and ρ. Also, define the functions ζ_i, i = 1, 2 postulated in the theorem by ζ₁(q) = ρ(q) cos θ(q), ζ₂(q) = ρ(q) sin θ(q). Then, if

${\omega }^1={\displaystyle \sum _{i=1}^2{\zeta }_i\left(\rho \left(q\right),\theta \left(q\right)\right)d{q}_i,}$

$W={\displaystyle {\int }_C{\omega }^1}$, and q : [0, 1] → M is a parametrization of C such that q(0) = q₀, q(1) = q, then

(i)  W is given by

$W\left(q,h\right)=\gamma {\displaystyle {\int }_0^1\sqrt{\left(h-V\left(q(s\right)\right)}}\left[\cos \theta \left(q\left(s\right)\right){q^{\prime }}_1\left(s\right)+\sin \theta \left(q\left(s\right)\right){q^{\prime }}_2\left(s\right)\right]ds$

(13.62)

where

$\text{γ=}\pm \sqrt{2}and{q}_i^{\prime }=\frac{d{q}_i\left(s\right)}{ds}.$

(ii)  W satisfies the HJE (13.59).

Proof: (i) If (13.60) is solvable for the function θ, then substituting the functions ζ_i(ρ(q), θ(q)), i = 1, 2 as defined above in (13.61), we get immediately

$\rho \left(q\right)=\pm \sqrt{2\left(h-V\left(q\right)\right)}.$

Further, by Theorem 13.3.1, ω¹ given above is exact, and $W={\displaystyle {\int }_C{\omega }^1}dq$ is independent of the path C. Therefore, if we parametrize the path C by s, then the above line integral can be performed coordinate-wise with W given by (13.62) and $\gamma =\pm \sqrt{2}.$ (ii) follows from Theorem 13.3.1. □

Remark 13.3.1 The above corollary gives one explicit parametrization that may be used. However, because the number of parameters available in the parametrization are limited, the above parametrization is only suitable for systems with n = 2. Other types of parametrizations that are suitable could also be employed.

If however the dimension n of the system is 3, then the following corollary gives another parametrization for solving the HJE.

Corollary 13.3.2 Assume the dimension n of the system is 3, and M, ρ, are as in the hypotheses of Theorem 13.3.1. Let ζ_i : ℜ × ℜ × ℜ → ℜ,i = 1, 2, 3 be defined by ζ₁(q) = ρ(q) sin θ(q) cos φ(q), ζ₂(q) = ρ(q) sin θ(q) sin φ(q), ζ₃(q) = ρ(q) cos θ(q), and assume (13.60) are solvable for θ and φ, while (13.61) is solvable for ρ. Then, if

${\omega }^1={\displaystyle \sum _{i=1}^3{\zeta }_i\left(\rho \left(q\right),\theta ,\phi \right)d{q}_i},$

$W={\displaystyle {\int }_C{\omega }^1}$, and q : [0, 1] → M is a parametrization of C such that q(0) = q₀, q(1) = q, then

(i)   W is given by

$\begin{array}{l}W\left(q,h\right)=\gamma {\displaystyle {\int }_0^1\sqrt{\left(h-V\left(q\left(s\right)\right)\right)}}\{\sin \theta \left(q\left(s\right)\right)\cos \phi \left(q\left(s\right)\right){q^{\prime }}_1\left(s\right)+\\ \sin \theta \left(q\left(s\right)\right)sin\phi \left(q\left(s\right)\right){q^{\prime }}_2\left(s\right)+\cos \theta \left(q\left(s\right)\right){q^{\prime }}_3\left(s\right)\}ds,\end{array}$

(13.63)

where $\gamma =\pm \sqrt{2}.$

(ii)  W satisfies the HJE (13.59).

Proof: Proof follows along the same lines as Corollary 13.3.1. □

Remark 13.3.2 Notice that the parametrization employed in the above corollary is now of a spherical nature.

If the HJE (13.59) is solvable, then the dynamics of the system evolves on the n-dimensional Lagrangian-submanifold $\tilde{N}$ which is an immersed-submanifold of maximal dimension, and can be locally parametrized as the graph of the function W, i.e.,

$\tilde{N}=\left\{\left(q,\frac{\partial W}{\partial q}\right)|q\in N\subset M,W\text{is}\text{a solution of}\text{ HJE}\left(13.59\right)\right\}.$

Moreover, for any other solution $W^{\prime }$ of the HJE, the volume enclosed by this surface is invariant. This is stated in the following proposition.

Proposition 13.3.1 Let N ⊂ M be the region in M where the solution W of the HJE (13.59) exists. Then, for any orientation of M, the volume-form of $\tilde{N}$

${\omega }^n=\left(\sqrt{1+{\displaystyle \sum _{j=1}^n{\left(\frac{\partial W}{\partial q_j}\right)}^2}}\right)d{q}_1\wedge d{q}_2\dots \wedge d{q}_n$

is given by

${\omega }^n=\left(\sqrt{1+2\left(h-V\left(q\right)\right)}\right)d{q}_1\wedge d{q}_2\dots \wedge d{q}_n.$

Proof: From the HJE (13.59), we have

$\begin{array}{l}\sqrt{1+{\displaystyle \sum _{j=1}^n{\left(\frac{\partial W}{\partial q_j}\right)}^2}}=\sqrt{1+2\left(h-V\left(q\right)\right)},\forall q\in N\\ \Updownarrow \\ {\omega }^n=\left(\sqrt{1+{\displaystyle \sum _{j=1}^n{\left(\frac{\partial W}{\partial q_j}\right)}^2}}\right)d{q}_1\wedge \dots \wedge d{q}_n=\left(\sqrt{1+2\left(h-V\left(q\right)\right)}\right)d{q}_1\wedge \dots \wedge d{q}_n.\square \end{array}$

We now apply the above ideas to solve the HJE for the two-particle nonperiodic ${\mathcal{A}}_2$ Toda-lattice described in Chapter 4.

13.3.2   Solving the Hamilton-Jacobi Equation for the ${\mathcal{A}}_2$-Toda System

Recall the Hamiltonian function and the canonical equations for the nonperiodic Toda lattice from Chapter 4:

$H\left(q,p\right)=\frac{1}{2}{\displaystyle \sum _{j=1}^n{p}_j^2}+{\displaystyle \sum _{j=1}^{n-1}{e}^{2\left(q_j-q_j+1\right)}}.$

(13.64)

Thus, the canonical equations for the system are given by

$\begin{array}{l}\frac{d{q}_j}{dt}=p_{j}j=1,\mathrm{\dots },n,\\ \frac{d{p}_1}{dt}=-2e^{2\left(q_1-q_2\right)}\\ \frac{d{p}_j}{dt}=-2e^{2\left(q_j-q_j+1\right)}+2e^{2\left(q_{j-1}-q_j\right)},j=2,\mathrm{\dots },n-1,\\ \frac{d{p}_n}{dt}=2e^{2\left(q_{n-1}-q_n\right).}\end{array}\}$

(13.65)

Consequently, the two-particle system (or ${\mathcal{A}}_2$ system) is given by the Hamiltonian (13.64):

$H\left(q_1,q_2,p_1,p_2\right)=\frac{1}{2}\left(p_1^2+p_2^2\right)+e^{2\left(q_1-q_2\right)},$

(13.66)

and the HJE corresponding to this system is given by

$\frac{1}{2}\left\{{\left(\frac{\partial W}{\partial q_1}\right)}^2+{\left(\frac{\partial W}{\partial q_2}\right)}^2\right\}+e^{2\left(q_1-q_2\right)}=h_2.$

(13.67)

The following proposition then gives a solution of the above HJE corresponding to the ${\mathcal{A}}_2$ system.

Proposition 13.3.2 Consider the HJE (13.67) corresponding to the ${\mathcal{A}}_2$ Toda lattice. Then, a solution for the HJE is given by

$\begin{array}{l}W\left({q^{\prime }}_1,{q^{\prime }}_2,h_2\right)=\cos \left(\frac{\text{π}}{4}\right){\displaystyle {\int }_1^{{q^{\prime }}_1}\rho \left(q\right)d{q}_1+m\sin \left(\frac{\Pi }{4}\right){\displaystyle {\int }_1^{{q^{\prime }}_1}\rho \left(q\right)d{q}_1}}\\ =\left(1+m\right)\{\frac{\sqrt{h_2-e^{-2\left(b+m-1\right)}}-\sqrt{h_2}{\tanh }^{-1}\left[\frac{\sqrt{h_2-e^{-2\left(b+m-1\right)}}}{\sqrt{h_2}}\right]}{m-1}-\\ \frac{\sqrt{h_2-e^{-2b-2\left(m-1\right){q^{\prime }}_1}}-\sqrt{h_2}{\tanh }^{-1}\left[\frac{\sqrt{h_2-e^{-2b-2\left(m-1\right){q^{\prime }}_1}}}{\sqrt{h_2}}\right]}{m-1}\},q_1>q_2\end{array}$

and

$\begin{array}{l}W\left({q^{\prime }}_1,{q^{\prime }}_2,h_2\right)=\cos \left(\frac{\text{π}}{4}\right){\displaystyle {\int }_1^{q_1\text{'}}\rho \left(q\right)d{q}_1+m\sin \left(\frac{\text{π}}{4}\right){\displaystyle {\int }_1^{{q^{\prime }}_1}\rho \left(q\right)d{q}_1}}\\ =\left(1+m\right)\{\frac{\sqrt{h_2-e^{-2\left(b+m-1\right)}}-\sqrt{h_2}{\tanh }^{-1}\left[\frac{\sqrt{h_2-e^{-2\left(b+m-1\right)}}}{\sqrt{h_2}}\right]}{m-1}-\\ \frac{\sqrt{h_2-e^{-2b-2\left(m-1\right){q^{\prime }}_1}}-\sqrt{h_2}{\tanh }^{-1}\left[\frac{\sqrt{h_2-e^{-2b-2\left(m-1\right){q^{\prime }}_1}}}{\sqrt{h_2}}\right]}{m-1}\},q_2>q_1.\end{array}$

Furthermore, a solution for the system equations (13.65) for the ${\mathcal{A}}_2$ with the symmetric initial q₁(0) = −q₂(0) and ${\dot{q}}_1\left(0\right)={\dot{q}}_2\left(0\right)=0$ is

$q\left(t\right)=\frac{1}{2}\log \sqrt{h_2}+\frac{1}{2}\log \left[\cosh 2\sqrt{h_2}\left(\beta -t\right)\right],$

(13.68)

where h₂ is the energy constant and

$\beta =\frac{1}{2\sqrt{h_2}}{\tanh }^{-1}\left(\frac{2{\dot{q}}_1^2\left(0\right)}{\sqrt{2h_2}}\right).$

Proof: Applying the results of Theorem 13.3.1, we have $\frac{\partial W}{\partial q_1}=\rho \left(q\right)\cos \theta \left(q\right),\frac{\partial W}{\partial q_2}=\rho \left(q\right)\sin \theta \left(q\right).$

Substituting this in the HJE (13.67), we immediately get

$\rho \left(q\right)=\pm \sqrt{2\left(h_2-e^{2\left(q_1-q_2\right)}\right)}$

and

${\rho }_{q_2}\left(q\right)\cos \theta \left(q\right)-{\theta }_{q_2}\rho \left(q\right)\sin \theta \left(q\right)={\rho }_{q_1}\left(q\right)\sin \theta \left(q\right)+{\theta }_{q_1}\rho \left(q\right)\cos \theta \left(q\right).$

(13.69)

The above equation (13.69) is a first-order PDE in θ and can be solved by the method of characteristics developed in Chapter 4. However, the geometry of the system allows for a simpler solution. We make the simplifying assumption that θ is a constant function. Consequently, equation (13.69) becomes

${\rho }_{q_2}\left(q\right)\cos \theta ={\rho }_{q_1}\left(q\right)\sin \theta \Rightarrow \tan \theta =\frac{{\rho }_{q_2}\left(q\right)}{{\rho }_{q_1}\left(q\right)}=-1\Rightarrow \theta =-\frac{\text{π}}{4}.$

Thus,

$\begin{array}{l}{p}_1=\rho \left(q\right)\cos \left(\frac{\text{π}}{4}\right),\\ p_2=-\rho \left(q\right)\sin \left(\frac{\text{π}}{4}\right),\end{array}$

and integrating dW along the straightline path from (1, −1) on the line segment

$L:q_2=\frac{q_2^{\prime }+1}{q_1^{\prime }+1}{q}_1+\left(1+\frac{q_2^{\prime }+1}{q_1^{\prime }+1}\right)\triangleq m{q}_1+b$

to some arbitrary point $\left(q_1^{\prime },q_2^{\prime }\right)$

we get

$\begin{array}{l}W\left(q_1^{\text{'}},q_2^{\text{'}},h_2\right)=\cos \left(\frac{\text{π}}{4}\right){\displaystyle {\int }_1^{q_1^{\text{'}}}\rho \left(q\right)d{q}_1+m\sin \left(\frac{\text{π}}{4}\right){\displaystyle {\int }_1^{q_1^{\text{'}}}\rho \left(q\right)d{q}_1}}\\ =\left(1+m\right)\{\frac{\sqrt{h_2-e^{-2\left(b+m-1\right)}}-\sqrt{h_2}{\tanh }^{-1}\left[\frac{\sqrt{h_2-e^{-2\left(b+m-1\right)}}}{\sqrt{h_2}}\right]}{m-1}-\\ \frac{\sqrt{h_2-e^{-2b-2\left(m-1\right)q_1^{\text{'}}}}-\sqrt{h_2}{\tanh }^{-1}\left[\frac{\sqrt{h_2-e^{-2b-2\left(m-1\right)q_1^{\text{'}}}}}{\sqrt{h_2}}\right]}{m-1}\}.\end{array}$

Similarly, if we integrate from point (−1,1) to $\left(q_1^{\prime },q_2^{\prime }\right)$, we get

$\begin{array}{l}W\left(q_1^{\text{'}},q_2^{\text{'}},h_2\right)=\cos \left(\frac{\text{π}}{4}\right){\displaystyle {\int }_{-1}^{q_1^{\text{'}}}\rho \left(q\right)d{q}_1+m\sin \left(\frac{\text{π}}{4}\right){\displaystyle {\int }_{-1}^{q_1^{\text{'}}}\rho \left(q\right)d{q}_1}}\\ =\left(1+m\right)\{\frac{\sqrt{h_2-e^{-2\left(b+m-1\right)}}-\sqrt{h_2}{\tanh }^{-1}\left[\frac{\sqrt{h_2-e^{-2\left(b+m-1\right)}}}{\sqrt{h_2}}\right]}{m-1}-\\ \frac{\sqrt{h_2-e^{-2b+2\left(m-1\right)q_1^{\text{'}}}}-\sqrt{h_2}{\tanh }^{-1}\left[\frac{\sqrt{h_2-e^{-2b+2\left(m-1\right)q_1^{\text{'}}}}}{\sqrt{h_2}}\right]}{m-1}\}.\end{array}$

Finally, from (4.11) and (13.66), we can write

$\begin{array}{l}{\dot{q}}_1=p_1=\rho \left(q\right)\cos \left(\frac{\text{π}}{4}\right),\\ {\dot{q}}_2=p_2=-\rho \left(q\right)\sin \left(\frac{\text{π}}{4}\right).\end{array}$

Then ${\dot{q}}_1+{\dot{q}}_2=0$ which implies that q₁ + q₂ = k, a constant, and by our choice of initial conditions, k = 0 or q₁ = −q₂ = −q. Now integrating the above equations from t = 0 to t we get

$\begin{array}{l}\frac{1}{2\sqrt{h_2}}{\tanh }^{-1}\left(\frac{\rho \left(q\right)}{\sqrt{2h_2}}\right)=\frac{1}{2\sqrt{h_2}}{\tanh }^{-1}\left(\frac{\rho \left(q\left(0\right)\right)}{\sqrt{2h_2}}\right)-t,\\ \frac{1}{2\sqrt{h_2}}{\tanh }^{-1}\left(\frac{\rho \left(q\right)}{\sqrt{2h_2}}\right)=-\frac{1}{2\sqrt{h_2}}{\tanh }^{-1}\left(\frac{\rho \left(q\left(0\right)\right)}{\sqrt{2h_2}}\right)+t.\end{array}$

Then, if we let

$\beta =\frac{1}{2\sqrt{h_2}}{\tanh }^{-1}\left(\frac{\rho \left(q\left(0\right)\right)}{\sqrt{2h_2}}\right),$

and upon simplification, we get

$\begin{array}{l}{q}_1-q_2=\frac{1}{2}\log \left[h_2\left(1-{\tanh }^{2}2\sqrt{h_2}\left(\beta -t\right)\right)\right]\\ =\frac{1}{2}\log \left[h_2\sec h^{2}2\sqrt{h_2}\left(\beta -t\right)\right].\end{array}$

(13.70)

Therefore,

$\begin{array}{l}q\left(t\right)=-\frac{1}{2}\log \sqrt{h_2}-\frac{1}{2}\log \left[\sec h2\sqrt{h_2}\left(\beta -t\right)\right]\\ =-\frac{1}{2}\log \sqrt{h_2}+\frac{1}{2}\log \left[\cos h2\sqrt{h_2}\left(\beta -t\right)\right].\end{array}$

Now, from (13.67) and (13.70), (13.70),

$\rho \left(q\left(0\right)\right)={\dot{q}}_1^2\left(0\right)+{\dot{q}}_2^2\left(0\right),$

and in particular, if ${\dot{q}}_1\left(0\right)={\dot{q}}_2\left(0\right)=0$ then β = 0. Consequently,

$q\left(t\right)=-\frac{1}{2}\log \sqrt{h_2}+\frac{1}{2}\log \left(\cos h2\sqrt{h_{2}t}\right)$

(13.71)

which is of the form (4.57) with $v=\sqrt{h.}$ □

13.4  Notes and Bibliography

The material of Section 13.1 is based on the References [193, 112, 264], and practical application of the approach to a flexible robot link can be found in the Reference [286]. While the material of Section 13.1.1 is based on Reference [71]. Similar approach using Galerkin’s approximation can be found in [63], and an extension of the approach to the DHJIE can also be found in [125].

The factorization approach presented in Section 13.2 is based on the author’s contribution [10], and extension of the approach to stochastic systems can be found also in [11]. This approach is promising and is still an active area of research.

Furthermore, stochastic HJBEs are extensively discussed in [287, 98, 91, 167], and numerical algorithms for solving them can be found in [173].

Various practical applications of nonlinear ${\mathcal{H}}_{\infty }$-control and methods for solving the HJIE can be found in the References [46, 80, 87, 154, 155, 286]. An alternative method of solving the HJIE using backstepping is discussed in [96], while other methods of optimally controlling affine nonlinear systems that do not use the HJE, in particular using inverse-optimality, can be found in [91, 167]-[169].

The material of Section 13.3 is also solely based on the author’s contribution [16]. More general discussions of HJE applied to mechanical systems can be found in the well-edited books [1, 38, 127, 115, 122, 200]. While the Toda lattice is discussed also extensively in the References [3]-[5], [72, 123, 201].

Generalized and viscosity solutions of Hamilton-Jacobi equations which may not necessarily be smooth are discussed extensively in the References [56, 83, 95, 97, 98, 186].

Finally, we remark that solving HJEs is in general still an active area of research, and many books have been written on the subject. There is not still a single approach that could be claimed to be satisfactory, all approaches have their advantages and disadvantages.

¹A set of eigenvalues {λ₁,…, λ_n} is said to be resonant if ${\sum }_{j=1}^{n}ij{\lambda }_j=0$ for some nonnegative integers i₁, i₂,…, i_n in such that ${\sum }_{j=1}^{n}ij>0.$ Otherwise, it is called nonresonant.

²A subset of a set is simply connected if a loop inside it can be continuously shrinked to a point.

³A 1-form σ : TM →ℜ is closed if dσ = 0. It is exact if σ = dS for some smooth function S : M →ℜ.