Quadratic Approximate Dynamic Programming for Input-Affine Systems

International Journal of Robust and Nonlinear Control, 24(3):432-449, February 2014.

• quad_adp_inp_aff.pdf

We consider the use of quadratic approximate value functions for stochastic control problems with input-affine dynamics and convex stage cost and constraints. Evaluating the approximate dynamic programming policy in such cases requires the solution of an explicit convex optimization problem, such as a quadratic program, which can be carried out efficiently. We describe a simple and general method for approximate value iteration, that also relies on our ability to solve convex optimization problems, in this case typically a semidefinite program. While we have no theoretical guarantee on the performance attained using our method, we observe that very good performance can be obtained in practice.