Viscosity Solutions of the Bellman Equation for Exit Time Optimal Control Problems with Non-Lipschitz Dynamics

Department of Systems Science and Mathematics, Washington University in Saint Louis, One Brookings Drive, Campus Box 1040, Saint Louis, MO 63130-4899, U.S.A.; malisoff@zach.wustl.edu.

Received: 19 April 2000
Revised: 22 December 2000

Abstract

We study the Bellman equation for undiscounted exit time optimal control problems with fully nonlinear Lagrangians and fully nonlinear dynamics using the dynamic programming approach. We allow problems whose non-Lipschitz dynamics admit more than one solution trajectory for some choices of open loop controls and initial positions. We prove a uniqueness theorem which characterizes the value functions of these problems as the unique viscosity solutions of the corresponding Bellman equations that satisfy appropriate boundary conditions. We deduce that the value function for Sussmann's Reflected Brachystochrone Problem for an arbitrary singleton target is the unique viscosity solution of the corresponding Bellman equation in the class of functions which are continuous in the plane, null at the target, and bounded below. Our results also apply to degenerate eikonal equations, and to problems whose targets can be unbounded and whose Lagrangians vanish for some points in the state space which are outside the target, including Fuller's Example.