The problem classes presented in this section provide characteristic methodologies that can be exploited in broader optimal control problems, some of which have been exploited in this book. The rest can be used in various extensions to the study of intelligent attack strategies. In general, the Fundamental Theorem of Calculus of Variations can be exploited in optimal control theory for finding trajectories that minimize performance measures expressed as functionals. This is also the case in the corresponding modeling of malware diffusion in
Chapter 6 and potential extensions to attack strategies.
C.3.1. Functionals of a Single Function
When the performance metric is a function of a single scalar function
x in the class of continuous first derivatives, the functional
can be written in the form
J(x)=∫tft0g(x(t),ẋ (t),t)dt.
(C.14)
The objective is to find
x∗ for which
J(x) has a relative extremum. Functions (curves) in
Ω that satisfy the end conditions are called admissible.
When
x(t0),
x(tf) are free and
tf specified, the functional increment can be written as
ΔJ(x,δx)=∫tft0[g(x(t)+δx(t),ẋ (t)+δẋ (t),t)−g(x(t),ẋ (t),t)]dt,
(C.15)
which using a Taylor series expansion about
(x(t),ẋ (t)) can be expressed as
δJ(x,δx)=∫tft0{[∂g∂x(x(t),ẋ (t),t)]δx(t)+[∂g∂ẋ (x(t),ẋ (t),t)]δẋ (t)}dt.
(C.16)
By the fundamental theorem of calculus of variations, a necessary condition for
x∗ to be an extremal is
∂g∂x(x∗(t),ẋ ∗(t),t)−ddt[∂g∂ẋ (x∗(t),ẋ ∗(t),t)]=0,∀t∈[t0,tf].
(C.17)
The latter is an Euler differential equation. More specifically, it is a second order nonlinear ordinary and time-varying differential equation, which is hard to solve since it yields a two-point boundary-value problem with split boundary conditions.
When an extremal for the metric given in
(C.15) with
t0,tf,x(t0) specified and
x(tf) free (which corresponds to the case where one boundary condition is given and the second is obtained from Euler equation-differential boundary condition) is required, the same approach as before applies as well, where now the metric functional is integrated by parts. It turns out that the necessary condition for
x∗ to be an extremal is Eq.
(C.17), and we may also obtain a natural boundary condition
∂g∂ẋ (x∗(tf),ẋ ∗(tf),tf)=0.
(C.18)
Thus, an extremal for a free endpoint problem is also an extremal for the fixed endpoint problem with the same endpoints and the same functional.
If one is given metric
(C.15) and
t0,x(t0)=x0,x(tf)=xf are specified while
tf is free, Euler condition
(C.17) is satisfied, along with the boundary condition
g(x∗(tf),ẋ ∗(tf),tf)−[∂g∂ẋ (x∗(tf),ẋ ∗(tf))]ẋ ∗(tf)=0.
Finally, when an extremal is sought for metric
(C.15) with
t0,x(t0)=x0 specified and
tf,x(tf)=xf is free, again Euler condition
(C.17) is satisfied, and when
tf and
x(tf) are unrelated,
δxf,δtf are independent and arbitrary yielding
g(x∗(tf),ẋ ∗(tf),tf)=0, while if
tf−x(tf) are related, e.g. in the form
x(tf)=θ(tf), then
δxf=∂θ∂t(tf)δtf and the transversality condition
[∂g∂ẋ (x∗(tf),ẋ ∗(tf),tf)][dθdt(tf)−ẋ ∗(tf)]+g(x∗(tf),ẋ ∗(tf),tf)=0
(C.19)
is obtained for solving the problem.
C.3.4. Constrained Extrema
State-control equations involve finding extrema of functionals of
n+m functions, namely, the state
x and controls
u, where only
m of the functions are independent, i.e. the controls
u. For the constrained minimization of functions, two alternative methodologies can be used, namely, the elimination method and the Lagrange multipliers method. The first can be employed if the dependence of variables/functions can be solved with respect to one of them and substituted in the differential
df(y∗)=0.
More specifically, assume the problem of finding the extreme values for a function of
n+m variables
y1,...,yn+mf(y1,...,yn+m), given
n constraints of the form
ai(y1,...,yn+m)=0,i=1,2,⋯,n,
where only
m variables out of the
n+m are independent.
Under the elimination method and the above constraints,
yi=ei(y1,...,yn+m)=0,i=1,2,⋯.n,
and then by substituting into
f, a function
f(yn+1,...,yn+m) of
m independent variables is obtained. Then, this satisfies the following system of equations:
∂f∂yn+1(y∗n+1,...,y∗n+m)=0,
∂f∂yn+m(y∗n+1,...,y∗n+m)=0,
which can be solved for
y∗n+1,...,y∗n+m and substituted back in
y1,...,yn to obtain
y∗1,...,y∗n. Then
f(y∗1,...,y∗n+m) can be obtained.
In the Lagrange multipliers method, the augmented function
fa(y1,...,yn+m,p1,...,pn)=f(y1,...,yn+m)+p1[a1(y1,...,yn+m)]+...+pn[an(y1,...,yn+m)]
(C.23)
is defined and then the differential of the augmented function is obtained
dfa=∂fa∂y1Δy1+...∂fa∂yn+mΔyn+m+∂fa∂p1Δp1+...+∂fa∂p1Δpn=∂fa∂y1Δy1+...∂fa∂yn+mΔyn+m+a1Δp1+...anΔpn,
from which
2n+m equations from the KKT conditions (
[27,
42])
ai(y∗1,...,y∗n+m)=0,i=1,2,...,n,
∂fa∂yj(y∗1,....,y∗n+m,p∗1,...p∗n)=0,j=1,2,...,n+m
can be obtained, leading to
(y∗1,...,y∗n+m).
The above are valid for functions of single or multiple variables. In case of minimization of functionals, however, additional constraints in the form of differential equation or point constraints are required, both of which lead to the necessary condition
∂ga∂w(w∗(t),ẇ ∗(t),p∗(t),t)−ddt[∂ga∂ẇ (w∗(t),ẇ ∗(t),p∗(t),t)]=0,
(C.24)
where
ga(w(t),ẇ (t),p(t),t)=g(w(t),ẇ (t),t)+pT(t)[f(w(t),ẇ (t),t)].
Then the general form of the problem is to minimize the functional
J(w)=∫tft0g(w(t),ẇ (t),t)dt and the problem can be eventually solved using either the elimination or the Lagrange multipliers (augmented functional) method.
For the case of point constraints, the optimal
w∗ is required to satisfy constraints of the form
fi(w(t),t)=0,i=1,2,...,n, where only
m out of
n+m components of
w are independent. The augmented functional is
Ja(w,p)=∫tft0{g(w(t),ẇ (t),t)+p1(t)[f1(w(t),t)]+...+pn(t)[fn(w(t),t)]}dt=∫tft0[g(w,ẇ (t),t)+pT(t)[f(w(t),t)]dt.
On an extremal, the point constraints must be satisfied
f(w(t),t)=0, as well as
δJa(w∗,p)=0, where
δJa(w,δw,p,δp)=∫tft0{[∂gT∂w(w(t),ẇ (t),t)+pT(t)[∂f∂w(w(t),t)]]δw(t)+[∂gT∂w(w(t),ẇ (t),t)]δẇ (t)+[fT(w(t),t)]δp(t)}dt,
leading to aforementioned condition
(C.24).
In the cases when the constraints are given in the form of differential equations
fi(w(t),ẇ (t),t)=0,i=1,2,...,n, where again only
m out of
n+m components of
w are independent. In this case, the elimination approach is almost impossible, and Lagrange multipliers have to be adopted. The augmented functional is as before and the conditions on the extremals
f(w∗(t),ẇ ∗(t),t)=0 and
δJa(w∗,p)=0 again yield condition
(C.24), where
δJa(w,δw,p,δp)=∫tft0{[∂gT∂w(w(t),ẇ (t),t)+pT(t)[∂f∂w(w(t),ẇ (t),t)]]δw(t)+[∂gT∂ẇ (w(t),ẇ (t),t)+pT(t)[∂f∂ẇ (w(t),ẇ (t),t)]]δẇ (t)+[fT(w(t),ẇ (t),t)]δp(t)}dt.
Finally, the extremal
w∗ for the functional could be subject to isoperimetric constraints of the form
∫tft0ei(w(t),ẇ (t),t)dt=ci,i=1,2,...,n. In this case, the auxiliary variable
zi=∫tft0ei(w(t),ẇ (t),t)dt=ci is defined, so that in vector form
z(t)=e(w(t),ẇ (t),t) with boundary values
zi(t0)=0 and
zi(tf)=ci. In this case, the augmented functional becomes
ga(w(t),ẇ (t),p(t),t)=g(w(t),ẇ (t),t)+pT(t)[e(w(t),ẇ (t),t)−ż (t)].
Then, the overall system can be solved by using the
n+m available equations obtained from
∂ga∂w(w∗(t),ẇ ∗(t),p∗(t),ż ∗(t),t)−ddt[∂ga∂ẇ (w∗(t),ẇ ∗(t),p∗(t),ż ∗(t),t)]=0,
the
r equations available from the following:
∂ga∂z(w∗(t),ẇ ∗(t),p∗(t),ż ∗(t),t)−ddt[∂ga∂ż (w∗(t),ẇ ∗(t),p∗(t),ż ∗(t),t)]=0
and the
r equations available from
z(t)=e(w(t),ẇ (t),t),
while in addition it holds that
ṗ (t)=0.