Hamiltonian (control theory)
Encyclopedia
The Hamiltonian of optimal control theory
was developed by L. S. Pontryagin as part of his minimum principle
. It was inspired by, but is distinct from, the Hamiltonian
of classical mechanics. Pontryagin proved that a necessary condition for solving the optimal control problem is that the control should be chosen so as to minimize the Hamiltonian. For details see Pontryagin's minimum principle
.
is to be chosen so as to minimize the objective function:
The system state
evolves according to the state equations
the control must satisfy the constraints
Optimal control
Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. The method is largely due to the work of Lev Pontryagin and his collaborators in the Soviet Union and Richard Bellman in the United States.-General method:Optimal...
was developed by L. S. Pontryagin as part of his minimum principle
Pontryagin's minimum principle
Pontryagin's maximum principle is used in optimal control theory to find the best possible control for taking a dynamical system from one state to another, especially in the presence of constraints for the state or input controls. It was formulated by the Russian mathematician Lev Semenovich...
. It was inspired by, but is distinct from, the Hamiltonian
Hamiltonian mechanics
Hamiltonian mechanics is a reformulation of classical mechanics that was introduced in 1833 by Irish mathematician William Rowan Hamilton.It arose from Lagrangian mechanics, a previous reformulation of classical mechanics introduced by Joseph Louis Lagrange in 1788, but can be formulated without...
of classical mechanics. Pontryagin proved that a necessary condition for solving the optimal control problem is that the control should be chosen so as to minimize the Hamiltonian. For details see Pontryagin's minimum principle
Pontryagin's minimum principle
Pontryagin's maximum principle is used in optimal control theory to find the best possible control for taking a dynamical system from one state to another, especially in the presence of constraints for the state or input controls. It was formulated by the Russian mathematician Lev Semenovich...
.
Notation and Problem statement
A control is to be chosen so as to minimize the objective function:The system state
evolves according to the state equationsthe control must satisfy the constraints
Definition of the Hamiltonian
-
where
is a vector of costate variables of the same dimension as the state variables 
.
For information on the properties of the Hamiltonian, see Pontryagin's minimum principlePontryagin's minimum principlePontryagin's maximum principle is used in optimal control theory to find the best possible control for taking a dynamical system from one state to another, especially in the presence of constraints for the state or input controls. It was formulated by the Russian mathematician Lev Semenovich...
.
The Hamiltonian in discrete time
When the problem is formulated in discrete time, the Hamiltonian is defined as:
and the costate equations are
(Note that the discrete time Hamiltonian at time
involves the costate variable at time 
This small detail is essential so that when we differentiate with respect to 
we get a term involving 
on the right hand side of the costate equations. Using a wrong convention here can lead to incorrect results, i.e. a costate equation which is not a backwards difference equation).
The Hamiltonian of control compared to the Hamiltonian of mechanics
William Rowan HamiltonWilliam Rowan HamiltonSir William Rowan Hamilton was an Irish physicist, astronomer, and mathematician, who made important contributions to classical mechanics, optics, and algebra. His studies of mechanical and optical systems led him to discover new mathematical concepts and techniques...
defined the HamiltonianHamiltonian mechanicsHamiltonian mechanics is a reformulation of classical mechanics that was introduced in 1833 by Irish mathematician William Rowan Hamilton.It arose from Lagrangian mechanics, a previous reformulation of classical mechanics introduced by Joseph Louis Lagrange in 1788, but can be formulated without...
as a function of three variables:
where
is defined implicitly by
Hamilton then formulated his equations as
In contrast the Hamiltonian of control theory (as defined by Pontryagin) is a function of 4 variables
and the associated conditions for a maximum are
This difference is somewhat confusing, nevertheless a specific problem, such as the Brachystochrone problem, can be solved by either method. For details, see the article by Sussmann and Willems.