This book provides a comprehensive overview of numerical methods for stochastic control problems in continuous time. It covers topics such as stochastic mechanics, random media, signal processing, image synthesis, mathematical economics, finance, stochastic modeling, and applied probability. The book also discusses stochastic optimization, control, and related areas. It includes references to various mathematical texts and research papers that are relevant to the subject. The book is divided into 16 chapters, starting with an introduction and a review of continuous time models. It then moves on to controlled Markov chains, dynamic programming equations, and the Markov chain approximation method. The following chapters discuss computational methods for controlled Markov chains, the ergodic cost problem, heavy traffic and singular control, weak convergence and characterization of processes, convergence proofs, and convergence for reflecting boundaries, singular control, and ergodic cost problems. The book also covers finite time problems and nonlinear filtering, controlled variance and jumps, problems from the calculus of variations with finite and infinite time horizons, and the viscosity solution approach. It includes detailed numerical methods and algorithms for solving stochastic control problems, as well as convergence theorems and error bounds. The book is intended for researchers and practitioners in the fields of stochastic control, optimization, and applied probability. It provides a thorough treatment of the subject, with a focus on numerical methods and their applications. The book is well-organized and includes a comprehensive list of references and an index for easy reference.This book provides a comprehensive overview of numerical methods for stochastic control problems in continuous time. It covers topics such as stochastic mechanics, random media, signal processing, image synthesis, mathematical economics, finance, stochastic modeling, and applied probability. The book also discusses stochastic optimization, control, and related areas. It includes references to various mathematical texts and research papers that are relevant to the subject. The book is divided into 16 chapters, starting with an introduction and a review of continuous time models. It then moves on to controlled Markov chains, dynamic programming equations, and the Markov chain approximation method. The following chapters discuss computational methods for controlled Markov chains, the ergodic cost problem, heavy traffic and singular control, weak convergence and characterization of processes, convergence proofs, and convergence for reflecting boundaries, singular control, and ergodic cost problems. The book also covers finite time problems and nonlinear filtering, controlled variance and jumps, problems from the calculus of variations with finite and infinite time horizons, and the viscosity solution approach. It includes detailed numerical methods and algorithms for solving stochastic control problems, as well as convergence theorems and error bounds. The book is intended for researchers and practitioners in the fields of stochastic control, optimization, and applied probability. It provides a thorough treatment of the subject, with a focus on numerical methods and their applications. The book is well-organized and includes a comprehensive list of references and an index for easy reference.