Scribd
Upload
46 views3 pages

Optimal Control Theory

University Mathematics is 100% positioned to meet the demands of undergraduate, postgraduate and professional exams. The book is capable and remarkable, giving students a good grounding in real-world mathematical applications. Inside you will find: • Simplified explanations step by step. • Solved examples of varying difficulties. • Plenty of workouts, fill-in-the-blanks and multiple-choice questions. Every student and every classroom will vastly benefit from University Mathematics

Uploaded by

Olaniyi Evans
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
46 views3 pages

Optimal Control Theory

University Mathematics is 100% positioned to meet the demands of undergraduate, postgraduate and professional exams. The book is capable and remarkable, giving students a good grounding in real-world mathematical applications. Inside you will find: • Simplified explanations step by step. • Solved examples of varying difficulties. • Plenty of workouts, fill-in-the-blanks and multiple-choice questions. Every student and every classroom will vastly benefit from University Mathematics

Uploaded by

Olaniyi Evans
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

25

OPTIMAL CONTROL THEORY

CONTENTS
The Hamiltonian & the Pontryagin’s Maximum Principle 338
Constraints in the Endpoints 343
The Current-Value Hamiltonian 344

Optimal control theory, an extension of the calculus of variations, is a branch of


mathematics that deals with finding the best way to control a dynamic system
over time such that an objective function is optimized. It is useful for solving
dynamic optimization problems expressed in continuous time. The system being
controlled is typically described by a set of differential equations, and the control
policy is a function that specifies the inputs to the system over time.
For example, the dynamical system might be a nation's economy, with the
objective to minimize inflation; the controls in this case could be interest rate
and money supply. For example, the dynamical system might be a firm's flow of
profits over time, with the objective to maximize the total profits; the controls in
this case could be the output and price.
Largely based on the work of Lev Pontryagin and Richard Bellman in the 1950s,
the optimal control theory has numerous applications in science, engineering
and operations research. It is used to design controllers for systems such as
aircraft, spacecraft, and robots, and to optimize resource allocation in industries
such as energy and transportation.
A control problem includes a functional that is a function of state
and control variables. An optimal control is a set of differential equations that
describe the paths of the control variables that minimize the functional.
The optimal control is derivable using Pontryagin's maximum principle (a
necessary condition), or by solving the Hamilton-Jacobi-Bellman equation (a
sufficient condition). In optimal control theory, the objective is to find the
optimal time path for the control variable, y. The state variable, x, helps us
to predict the behavior of the control variable and has equations of motion
(or transition) set equal to x’.
Optimal control theory problems could involve continuous time, a finite time
horizon, and fixed endpoints. A standard optimal control problem is of the form:
338 Olaniyi Evans | University Mathematics

𝑇
Maximize 𝑈 = ∫0 𝑓[𝑥(𝑡), 𝑦(𝑡), 𝑡] 𝑑𝑡
subject to 𝑥′ = 𝑔[𝑥(𝑡), 𝑦(𝑡), 𝑡] (1)
𝑥(0) = 𝑥0 𝑥(𝑇) = 𝑥𝑇
where U is the functional to be optimized; y(t) the control variable selected or
controlled to optimize U; x(t) the state variable which varies over time consistent
with the differential equation set equal to x’ in the constraint; and t time.

T H E H A M I LT O N IA N & T H E P O NT R Y A G I N ’ S
M A X IM U M P R I NC I P LE
The Hamiltonian function, similar to the Lagrangian function, is a technique
that combines the functional (being optimized) with the state variable
(describing the constraint or constraints) into a single equation.
In line with (1), the Hamiltonian is
𝐻[𝑥(𝑡), 𝑦(𝑡), 𝜆(𝑡), 𝑡] = 𝑓[𝑥(𝑡), 𝑦(𝑡), 𝑡] + 𝜆(𝑡)𝑔[𝑥(𝑡), 𝑦(𝑡), 𝑡] (2)
where H denotes the Hamiltonian, 𝜆(t) is the costate variable (similar to the
Lagrangian multiplier) which represents the marginal value or shadow price of
the state variable x(t).

Pontryagin’s Maximum Principle


For maximization of the Hamiltonian in (2), the necessary conditions are derived
from the maximum principle:
𝜕𝐻
i. =0
𝜕𝑦
𝜕𝑥 𝜕𝐻
ii. = 𝑥′ = state equation
𝜕𝑡 𝜕𝜆
𝜕𝜆 𝜕𝐻
iii. = 𝜆′ = − costate equation
𝜕𝑡 𝜕𝑥
iv. 𝑥(0) = 0 𝑥(𝑇) = 𝑥𝑇 boundary condition
v. 𝜆(𝑇) = 0 transversality condition
The first three conditions are known as the maximum principle. The two
equations of motion in conditions (ii) and (iii) are referred to as the Hamiltonian
system or the canonical system. Condition (v) is applicable to a free endpoint
problem only.
For minimization of the Hamiltonian, multiply the objective function by -1.

The sufficient conditions


The sufficient conditions are fulfilled if:

1. The objective functional and the constraint are continuously differentiable


and jointly concave in x and y.
2. 𝜆(𝑡) ≥ 0, when the constraint is non-linear in x or y. If the constraint is
linear, 𝜆 can assume any sign.

A linear function is both concave and convex, but neither strictly concave
nor strictly convex. For a nonlinear function, the discriminant can test for
joint concavity.

Given the discriminant of the second-order derivatives of a function,

𝑓𝑥𝑥 𝑓𝑥𝑦
|𝐷| = | |
𝑓𝑦𝑥 𝑓𝑦𝑦
Chapter 25 | Opimal Control Theory 339

Purchase the full book at:


https://unimath.5profz.com/

We donate 0.5% of the book sales


every year to charity, forever. When
you buy University Mathematics (I &
II) you are helping orphans and the
less privileged.

You might also like