21 Introduction
The optimal control problems can be seen as a functional minimization, called target, subjected to ordinary differential constraints.
21.1 Isoperimetric problem
Let’s consider a function
y (x) \in [a,b]\mapsto\mathbb{R}
where the are known the point y(a) = y(b) = 0. If we consider the function as a rope having fixed length l that, by calculus I, can be computed as
l = \int_a^b \sqrt{1+y'(x)^2}\, \mathrm{d}x
As a first optimal-control problem, determine the function f that minimizes its integral over the prescribed domain:
\begin{aligned} \text{minimize:} \qquad & \mathcal{F}(y) = - \int_a^b y(x)\, \mathrm{d}x \\ \text{subject to:} \qquad & y(a) = 0,\quad y(b) = 0\\ & \int_a^b \sqrt{1+y'(x)^2}\, \mathrm{d}x - l = 0 \end{aligned}
21.2 Discretized solution
A way to solve the problem is by discretization, dividing the domain [a,b] in n edges having length h= \frac{b-a}{n} and so x_k = a + k h. By founding the discrete solution y_k of the system we can approximate also the continuous solution as
y_k \approx y(x_k)
Considering the integral approximation
\int_{x_{k-1}}^{x_k} y(x)\, \mathrm{d}x \approx h y_{k-\frac 1 2} = h \frac{y_{k} + y_{k-1}}{2}
and the derivative
y'(x_k) \approx y_{k-\frac 1 2}' = \frac{y_k - y_{k-1}}{2}
With this being stated the initial isoperimetric problem can be rewritten (with some analytical simplification for the calculus) in the discretized form as
\begin{aligned} \text{minimize:} \qquad & - \frac h 2\big(y_0+y_n\big) - h \sum_{k=1}^{n-1} y_k \\ \text{subject to:} \qquad & y_0 = 0,\quad y_n= 0\\ & l - h \sum_{k=1}^{n} \sqrt{1 + \left( \frac{y_k - y_{k-1}}{h} \right)^2} \end{aligned}
As here stated, this problems become a constrained minimization one that can so be solved using the Lagrange multiplier method on which the unknowns is the vector of the discretized function \bm{y} = (y_1,\dots, y_n) and the 3 Lagrange multipliers \lambda_0, \lambda_n, \lambda associated to the constraints:
\begin{aligned} \mathcal{L}\big(\bm{y},\lambda_0,\lambda_n,\lambda\big) & = - \frac{h}{2}\sum_{k=1}^{n} \big(y_k + y_{k-1}\big) - \lambda_0y_0 - \lambda_ny_n \\ & - \lambda \left( l - h \sum_{k=1}^{n} \sqrt{1 + \left( \frac{y_k - y_{k-1}}{h} \right)^2} \right) \end{aligned}
By calculating the gradient of the lagrangian and setting it to zero, we satisfy the first necessary condition for the solutions that relates to the following non-linear system that can be solved by approximate solutions:
for k=0:
- \frac{h}{2} - \lambda_0 + \lambda \frac{ \frac{y_1-y_0}{h} }{\sqrt{1- \left( \frac{y_1-y_0}{h} \right)^2}} = 0
for k = 1,2,\dots,n-1
-h + \frac{\lambda \frac{y_k-y_{k+1}}{h}}{\sqrt{1+\left( \frac{y_k - y_{k-1}}{h} \right)^2}} - \frac{\lambda \frac{y_{k+1}-y_{k}}{h}}{\sqrt{1+\left( \frac{y_{k+1} - y_{k}}{h} \right)^2}} = 0
for k=n:
- \frac{h}{2} - \lambda_n - \lambda \frac{\frac{y_n - y_0}{h}}{\sqrt{1 + \left( \frac{y_1-y_0}{h} \right)^2}} = 0
multipliers conditions:
\begin{cases} y_0 = 0,\qquad y_n = 0 \\ l-h\displaystyle\sum_{k=1}^{n} \sqrt{1 + \left( \frac{y_k - y_{k-1}}{h} \right)^2} = 0 \end{cases}
21.3 Continuous interpretation
Considering now the limit for h\to 0 the discretized systems seems to become continuous; in particular the 4-th conditions become
y_0 \to y(a) = 0 \quad \text{and} \quad y_n \to y(b) = 0
The first condition (associated to k=0) and the third (k = n) instead becomes
- \frac{h}{2} - \lambda_0 + \lambda \frac{ \frac{y_1-y_0}{h} } {\sqrt{1- \left( \frac{y_1-y_0}{h} \right)^2}} \quad\to\quad - \lambda_0 - \lambda \frac{y'(a)}{\sqrt{1-y'(a)^2}} = 0
- \frac{h}{2} - \lambda_n - \lambda \frac{\frac{y_n - y_0}{h}}{\sqrt{1 + \left( \frac{y_1-y_0}{h} \right)^2}} \quad\to\quad - \lambda(b) - \lambda \frac{y'(b)}{\sqrt{1-y'(b)^2}} = 0
With some analytical manipulation the second lagrange condition can be stated as
- 1 - \lambda \frac{\mathrm{d}}{\mathrm{d}x} \left. \frac{y'(x)}{\sqrt{1+y'(x)^2}} \right|_{x=x_k} = 0
21.4 Observation
By pushing the limit h\to 0 the discretized problem becomes continuous and can relate to the general formulation of minimization of a functional \mathcal{F} subjected to an equality constraints (depending on \mathcal G):
\begin{aligned} \text{minimize:} \quad & \mathcal{F}(y) = - \int_a^b y(x)\, \mathrm{d}x && \implies\int_a^b L(y,y',t)\,\mathrm{d}t \\ \text{subject to:} \quad & y(a) = 0, \quad y(b) = 0\\ & \int_a^b \sqrt{1+y'(x)^2}\, \mathrm{d}x - l = 0 && \implies\int_a^b \mathcal G(y,y',y)\, \mathrm{d}t = 0 \end{aligned}
In the isoperimetric problem we can determine the functions L(y,y',x)=-y whose derivative are
\frac{\partial L}{\partial y} = -1 \quad\text{and}\quad \frac{\partial L}{\partial y'} = 0
Using so the Euler-Lagrange approach we can see that it fails because
\frac{\partial L}{\partial y} - \frac{\mathrm{d}}{\mathrm{d}x} \frac{\partial L}{\partial y'} = - 1 \neq 0
Considering instead now the definition of the constraint
\mathcal{G}(y,y',x) = \frac{l}{b-a} - \sqrt{1 + y'^2}
we can build another lagrangian \tilde{ L } that also considers (with a Lagrange multiplier \lambda) the constraint:
\tilde{ L }(y,y',x,\lambda) = L(y,y',x) - \lambda \mathcal{G}(y,y',x)
Applying the Euler-Lagrange method on this expression it’s possible to get the same result achieved with discretization, in fact
\frac{\partial\tilde{L}}{\partial y} - \frac{\mathrm{d}}{\mathrm{d}x} \frac{\partial\tilde{L}}{\partial y'} = - 1 - \lambda \frac{y'}{\sqrt{1 + y'^2}}
21.5 General formulation from Euler-Lagrange
Considering the optimal control problem as
\begin{aligned} \text{minimize:} \qquad & \int_a^b L (x,x',t)\, \mathrm{d}t \\ \text{subject to:}\qquad & x(a) = x_a,\quad x(b) =x_b \\ & \int_a^b \mathcal G(x,x',t)\,\mathrm{d}t = 0 \end{aligned}
a way to reach a solution is by constructing the lagrangian \mathcal{L} defined as
\begin{aligned} \tilde{L}(x,x',t, \lambda) & = L(x,x',t) - \lambda \mathcal G(x,x',t) \\ \mathcal{L}(x,\lambda,\lambda_a,\lambda_b) & = \int_a^b \tilde{ L }(x,x',t,\lambda)\,\mathrm{d}t - \lambda_a x(a) - \lambda_b x(b) \end{aligned}
At this point the solution of the problem is the stationary point (that in reality is a function) x of the lagrangian, and this means setting to zero the Gateaux derivative of \mathcal{L}:
\begin{aligned} \delta\mathcal{L} &= \int_a^b \left( \frac{\partial\tilde{L}}{\partial x}\delta x + \frac{\partial\tilde{L}}{\partial x'} \delta x' + \frac{\partial\tilde{L}}{\partial \lambda} \delta\lambda \right)\, \mathrm{d}t \\ & - \delta\lambda_a \big(x(a)-x_a\big) - \lambda_a \delta x(a) - \delta\lambda_b \big(x(b)-x_b\big) - \lambda_b \delta x(b) \end{aligned}
Performing the integration by part in order to remove the term \delta x' the derivative so becomes
\begin{aligned} \delta \mathcal{L} & = \int_a^b A\,\delta x\, \mathrm{d}x - \delta\lambda_a \big(x(a)-x_a\big) - \delta\lambda_b\big(x(b)-x_b\big) \\ & + B \delta x(a) + C\delta x(b) \end{aligned}
where
A = \frac{\partial\tilde{L}}{\partial x} - \frac{\mathrm{d}}{\mathrm{d}t} \frac{\partial\tilde{L}}{\partial x'} \quad B = -\lambda_a - \frac{\partial\tilde{L}}{\partial y'}\Big|_a \quad C = -\lambda_b + \frac{\partial\tilde{L}}{\partial y'}\Big|_b
At this point the overall solution of the minimization problem becomes the following ordinary differential equation system:
\begin{cases} \left(\frac{\partial L}{\partial x} \lambda \frac{\partial\mathcal{G}}{\partial x} \right) - \frac{\mathrm{d}}{\mathrm{d}x} \left(\frac{\partial L}{\partial x'}- \lambda \frac{\partial\mathcal{G}}{\partial x'} \right) = 0 \\[0.5em] x(a) = x_a, \quad x(b) = x_b \\[0.5em] -\lambda_a - \left( \frac{\partial L}{\partial x'}\Big|_a- \lambda \frac{\partial\mathcal{G}}{\partial x'} \Big|_a \right) = 0\\[0.5em] -\lambda_b + \left(\frac{\partial L}{\partial x'}\Big|_b - \lambda \frac{\partial\mathcal{G}}{\partial x'} \Big|_b \right) = 0 \end{cases}
Ordinary differential equation constraint
Let’s consider the general formulation of the isoperimetric problem as
\begin{aligned} \text{minimize:} \qquad & \, \mathcal{F}(x) = \int_a^b L (x,x',t)\, \mathrm{d}t \\ \text{subject to:}\qquad & w\big(x(a),x(b) \big)= 0 \\ & \int_a^b \mathcal G(x,x',t)\,\mathrm{d}t = l \end{aligned}
the problem can be solved introducing the function z(t) such that z' = \mathcal{G}(x,x',t): this means that z(a) = 0 and z(b) = l and so the problem if transformed, considering also the introduction of the relation y = x', to the form
\begin{aligned} \text{minimize:} \qquad & \mathcal{F}(x,y) = \int_a^b L (x,y,t)\, \mathrm{d}t \\ \text{subject to:}\qquad & w\big(x(a),x(b) \big)= 0 \\ & z(a) = 0,\quad z(b) = l \\ & x' = y \\ & z' = \mathcal G(x,y,t) \end{aligned}
With the problem as here stated we refer to x and z as states, while y is the control. We can so see that an optimal control problem can be easily transform into a boundary value problem that can be solved using the technique shown on functional minimization.