Hamilton-jacobi-bellman Equation

The solution of the HJB equation is the 'value function', which gives the optimal cost-to-go for a given Dynamical System with an associated cost function. Classical variational problems, for example, the Brachistochrone Problem can be solved using this method as well.

The equation is a result of the theory of Dynamic Programming which was pioneered in the 1950s by Richard Bellman and coworkers. The corresponding discrete-time equation is usually referred to as the Bellman Equation . In continuous time, the result can be seen as an extension of earlier work in Classical Physics by William Rowan Hamilton and Carl Gustav Jacob Jacobi .

Consider the following problem in deterministic optimal control

:

\min \int_0^T C + D[x(T)

subject to

:

\dot{x}(t)=F

{Link without Title}

where

x(t)

is the system state,

x(0)

is assumed given, and

u(t)

for

0\leq t\leq T

is the control that we are trying to find.
For this simple system, the Hamilton Jacobi Bellman partial differential equation is

:

rac{\partial}{\partial t} V(x,t) + \min_u \left\{ \left\langle rac{\partial}{\partial x}V(x,t), F(x, u) 
ight
angle + C(x,u) 
ight\} = 0

subject to the terminal condition

:

V(x,T) = D(x).\,

The unknown

V(t, x)

in the above PDE is the Bellman 'value function', that is the cost incurred from starting in state

x

at time

t

and controlling the system optimally from then until time

T

.
The HJB equation needs to be solved backwards in time, starting from

t = T

and ending at

t = 0

.

The HJB equation is a sufficient condition for an optimum. If we can solve for

V

then we can find from it a control

u

that achieves the minimum cost.

The HJB method can be generalized to stochastic systems as well.

In general case, the HJB equation does not have a classical (smooth) solution. Several notions of generalized solutions have been developed to cover such situations, including Viscosity Solution ( Pierre-Louis Lions and Michael Crandall ), Minimax Solution ( Andrei Izmailovich Subbotin ), and others.

REFERENCES

R. E. Bellman. Dynamic Programming. Princeton, NJ, 1957.

	APPAREL
	BABY
	BEAUTY
	BOOKS
	CAR TOYS
	CELL PHONES
	DVD'S
	ELECTRONICS
	GOURMET FOOD
	GROCERIES
	HEALTH & PERSONAL
	HOME & GARDEN
	JEWELRY
	MUSIC
	MUSIC INSTRUMENTS
	OFFICE PRODUCTS
	SOFTWARE
	SPORTING GOODS
	TOOLS & HARDWARE
	TOYS
	VIDEO GAMES
	SHOPPING HOME

	MORE SHOPPING...

	CATEGORIES ABOUT HAMILTON-JACOBI-BELLMAN EQUATION

	control theory

	partial differential equations

Information About

Hamilton-jacobi-bellman Equation