Pontryagin's minimum principle
Encyclopedia
Pontryagin's maximum principle is used in optimal control
theory to find the best possible control for taking a dynamical system
from one state to another, especially in the presence of constraints for the state or input controls. It was formulated by the Russian mathematician Lev Semenovich Pontryagin
and his students. It has as a special case the Euler–Lagrange equation of the calculus of variations
.
The principle states informally that the Hamiltonian
must be minimized over
, the set of all permissible controls. If 
is the optimal control for the problem, then the principle states that:
where
is the optimal state trajectory and 
is the optimal costate trajectory.
The result was first successfully applied into minimum time problems where the input control is constrained, but it can also be useful in studying state-constrained problems.
Special conditions for the Hamiltonian can also be derived. When the final time
is fixed and the Hamiltonian does not depend explicitly on time 
, then:
and if the final time is free, then:
More general conditions on the optimal control are given below.
When satisfied along a trajectory, Pontryagin's minimum principle is a necessary condition for an optimum. The Hamilton–Jacobi–Bellman equation provides sufficient conditions for an optimum, but this condition must be satisfied over the whole of the state space.
to be the state of the dynamical system
with input
, such that
where
is the set of admissible controls and 
is the terminal (i.e., final) time of the system. The control 
must be chosen for all 
to minimize the objective functional 
which is defined by the application and can be abstracted as
The constraints on the system dynamics can be adjoined to the Lagrangian
by introducing time-varying Lagrange multiplier
vector
, whose elements are called the costates of the system. This motivates the construction of the Hamiltonian
defined for all 
by:
where
is the transpose of 
.
Pontryagin's minimum principle states that the optimal state trajectory
, optimal control 
, and corresponding Lagrange multiplier vector 
must minimize the Hamiltonian 
so that
for all time
and for all permissible control inputs 
. It must also be the case that
Additionally, the costate equations
must be satisfied. If the final state
is not fixed (i.e., its differential variation is not zero), it must also be that the terminal costates are such that
These four conditions in (1)-(4) are the necessary conditions for an optimal control. Note that (4) only applies when
is free. If it is fixed, then this condition is not necessary for an optimum.
Optimal control
Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. The method is largely due to the work of Lev Pontryagin and his collaborators in the Soviet Union and Richard Bellman in the United States.-General method:Optimal...
theory to find the best possible control for taking a dynamical system
Dynamical system
A dynamical system is a concept in mathematics where a fixed rule describes the time dependence of a point in a geometrical space. Examples include the mathematical models that describe the swinging of a clock pendulum, the flow of water in a pipe, and the number of fish each springtime in a...
from one state to another, especially in the presence of constraints for the state or input controls. It was formulated by the Russian mathematician Lev Semenovich Pontryagin
Lev Semenovich Pontryagin
Lev Semenovich Pontryagin was a Soviet mathematician. He was born in Moscow and lost his eyesight due to a primus stove explosion when he was 14...
and his students. It has as a special case the Euler–Lagrange equation of the calculus of variations
Calculus of variations
Calculus of variations is a field of mathematics that deals with extremizing functionals, as opposed to ordinary calculus which deals with functions. A functional is usually a mapping from a set of functions to the real numbers. Functionals are often formed as definite integrals involving unknown...
.
The principle states informally that the Hamiltonian
Hamiltonian (control theory)
The Hamiltonian of optimal control theory was developed by L. S. Pontryagin as part of his minimum principle. It was inspired by, but is distinct from, the Hamiltonian of classical mechanics. Pontryagin proved that a necessary condition for solving the optimal control problem is that the control...
must be minimized over
, the set of all permissible controls. If is the optimal control for the problem, then the principle states that:where
is the optimal state trajectory and is the optimal costate trajectory.The result was first successfully applied into minimum time problems where the input control is constrained, but it can also be useful in studying state-constrained problems.
Special conditions for the Hamiltonian can also be derived. When the final time
is fixed and the Hamiltonian does not depend explicitly on time , then:and if the final time is free, then:
More general conditions on the optimal control are given below.
When satisfied along a trajectory, Pontryagin's minimum principle is a necessary condition for an optimum. The Hamilton–Jacobi–Bellman equation provides sufficient conditions for an optimum, but this condition must be satisfied over the whole of the state space.
Maximization and minimization
The principle was first known as Pontryagin's maximum principle and its proof is historically based on maximizing the Hamiltonian. The initial application of this principle was to the maximization of the terminal velocity of a rocket. However as it was subsequently mostly used for minimization of a performance index it is also referred to as the minimum principle. Pontryagin's book solved the problem of minimizing a performance index.Notation
In what follows we will be making use of the following notation.Formal statement of necessary conditions for minimization problem
Here the necessary conditions are shown for minimization of a functional. Take to be the state of the dynamical systemDynamical system
A dynamical system is a concept in mathematics where a fixed rule describes the time dependence of a point in a geometrical space. Examples include the mathematical models that describe the swinging of a clock pendulum, the flow of water in a pipe, and the number of fish each springtime in a...
with input
, such thatwhere
is the set of admissible controls and is the terminal (i.e., final) time of the system. The control must be chosen for all to minimize the objective functional which is defined by the application and can be abstracted asThe constraints on the system dynamics can be adjoined to the Lagrangian
Lagrangian
The Lagrangian, L, of a dynamical system is a function that summarizes the dynamics of the system. It is named after Joseph Louis Lagrange. The concept of a Lagrangian was originally introduced in a reformulation of classical mechanics by Irish mathematician William Rowan Hamilton known as...
by introducing time-varying Lagrange multiplierLagrange multipliers
In mathematical optimization, the method of Lagrange multipliers provides a strategy for finding the maxima and minima of a function subject to constraints.For instance , consider the optimization problem...
vector
, whose elements are called the costates of the system. This motivates the construction of the HamiltonianHamiltonian (control theory)
The Hamiltonian of optimal control theory was developed by L. S. Pontryagin as part of his minimum principle. It was inspired by, but is distinct from, the Hamiltonian of classical mechanics. Pontryagin proved that a necessary condition for solving the optimal control problem is that the control...
defined for all by:where
is the transpose of .Pontryagin's minimum principle states that the optimal state trajectory
, optimal control , and corresponding Lagrange multiplier vector must minimize the Hamiltonian so thatfor all time
and for all permissible control inputs . It must also be the case thatAdditionally, the costate equations
must be satisfied. If the final state
is not fixed (i.e., its differential variation is not zero), it must also be that the terminal costates are such thatThese four conditions in (1)-(4) are the necessary conditions for an optimal control. Note that (4) only applies when
is free. If it is fixed, then this condition is not necessary for an optimum.See also
- Lagrange multipliers on Banach spacesLagrange multipliers on Banach spacesIn the field of calculus of variations in mathematics, the method of Lagrange multipliers on Banach spaces can be used to solve certain infinite-dimensional constrained optimization problems...
, Lagrangian method in calculus of variations