# Systems & Control: Foundations & Applications The book presents an up-to-date account of the theory of viscosity solutions of first-order partial differential equations of Hamilton-Jacobi type and its applications to optimal deterministic control and differential games. The theory of viscosity solutions, initiated in the 1980s by Crandall, Lions, and others, provides a convenient framework for dealing with the lack of smoothness of value functions in dynamic optimization problems. The main theme of the book is the application of the viscosity solutions approach to significant model problems in optimal control and differential games. The book emphasizes the advantages of this approach in establishing the well-posedness of Hamilton-Jacobi equations and its role in feedback synthesis when combined with techniques from optimal control theory and nonsmooth analysis. Chapter I introduces the main ideas using the infinite horizon discounted regulator problem as a model. It covers topics such as uniqueness, stability, and necessary and sufficient conditions for optimality. It also provides a quick review of discrete time and stochastic approximations to the value function. Chapter II presents the basic theory of continuous viscosity solutions, focusing on Hamilton-Jacobi-Bellman equations with convex Hamiltonians. It discusses the connections between viscosity solutions and various notions such as Lipschitz continuous functions, bilateral supersolutions, and extended solutions. Chapters III and IV specialize the basic theory to various optimal control problems with continuous value functions. Chapter III deals with problems with unrestricted state space, while Chapter IV focuses on problems involving exit times and state constraints. Chapter V discusses discontinuous value functions, including various notions of discontinuous viscosity solutions. It also presents the weak limits technique of Barles and Perthame, which is used in Chapters VI and VII. Chapter VI considers an approximation scheme for value functions based on discrete time Dynamic Programming. It also discusses regular perturbation problems. Chapter VII analyzes asymptotic problems such as singular perturbations, penalization of state constraints, vanishing discount, and vanishing switching costs. The limiting behavior of the associated Hamilton-Jacobi equations is analyzed using the viscosity solutions approach and the weak limit technique. Chapter VIII introduces the theory of two-person zero-sum differential games. It discusses different notions of value and the derivation of relevant Hamilton-Jacobi-Isaac equations. The viscosity solutions method is particularly useful for treating nonconvex, nonlinear PDEs. The book includes two appendices that deal with additional topics important for applications. The first appendix describes a computational method based on approximation theory. The second appendix provides a complete account of recent results on the viscosity solutions approach to H-infinity control. The book is intended as a self-contained and comprehensive presentation of the topic for scientists in the areas of optimal control, system theory, and partial differential equations. Each chapter includes a section of bibliographical and historical notes. The book originated from lecture notes of courses taught by the authors and is suitable for a graduate course in optimal control.# Systems & Control: Foundations & Applications The book presents an up-to-date account of the theory of viscosity solutions of first-order partial differential equations of Hamilton-Jacobi type and its applications to optimal deterministic control and differential games. The theory of viscosity solutions, initiated in the 1980s by Crandall, Lions, and others, provides a convenient framework for dealing with the lack of smoothness of value functions in dynamic optimization problems. The main theme of the book is the application of the viscosity solutions approach to significant model problems in optimal control and differential games. The book emphasizes the advantages of this approach in establishing the well-posedness of Hamilton-Jacobi equations and its role in feedback synthesis when combined with techniques from optimal control theory and nonsmooth analysis. Chapter I introduces the main ideas using the infinite horizon discounted regulator problem as a model. It covers topics such as uniqueness, stability, and necessary and sufficient conditions for optimality. It also provides a quick review of discrete time and stochastic approximations to the value function. Chapter II presents the basic theory of continuous viscosity solutions, focusing on Hamilton-Jacobi-Bellman equations with convex Hamiltonians. It discusses the connections between viscosity solutions and various notions such as Lipschitz continuous functions, bilateral supersolutions, and extended solutions. Chapters III and IV specialize the basic theory to various optimal control problems with continuous value functions. Chapter III deals with problems with unrestricted state space, while Chapter IV focuses on problems involving exit times and state constraints. Chapter V discusses discontinuous value functions, including various notions of discontinuous viscosity solutions. It also presents the weak limits technique of Barles and Perthame, which is used in Chapters VI and VII. Chapter VI considers an approximation scheme for value functions based on discrete time Dynamic Programming. It also discusses regular perturbation problems. Chapter VII analyzes asymptotic problems such as singular perturbations, penalization of state constraints, vanishing discount, and vanishing switching costs. The limiting behavior of the associated Hamilton-Jacobi equations is analyzed using the viscosity solutions approach and the weak limit technique. Chapter VIII introduces the theory of two-person zero-sum differential games. It discusses different notions of value and the derivation of relevant Hamilton-Jacobi-Isaac equations. The viscosity solutions method is particularly useful for treating nonconvex, nonlinear PDEs. The book includes two appendices that deal with additional topics important for applications. The first appendix describes a computational method based on approximation theory. The second appendix provides a complete account of recent results on the viscosity solutions approach to H-infinity control. The book is intended as a self-contained and comprehensive presentation of the topic for scientists in the areas of optimal control, system theory, and partial differential equations. Each chapter includes a section of bibliographical and historical notes. The book originated from lecture notes of courses taught by the authors and is suitable for a graduate course in optimal control.