EE365: Stochastic Control

Spring Quarter 2014

Course description

Introduction to stochastic control, with applications taken from a variety of areas including supply-chain optimization, advertising, finance, dynamic resource allocation, caching, and traditional automatic control. Markov decision processes, optimal policy with full state information for finite-horizon case, infinite-horizon discounted, and average stage cost problems. Bellman value function, value iteration, and policy iteration. Approximate dynamic programming. Linear quadratic stochastic control.

Prerequisites: Linear algebra (as in EE263) and probability (as in EE178 or MS&E220).


Professor Sanjay Lall and teaching assistants Samuel Bakouch, Alex Lemon and Paris Syminelakis.


  • EE365 is the same as MS&E251, Stochastic Decision Models.

  • Homework 8 solutions have been posted.

  • Last year's final for practice, and the solutions.

  • This year's final and solutions.

  • As a reminder, you are responsible for all announcements made on the Piazza forum.

  • Paris’ pre-final office hours: Thursday Jun 5, 11-1 in Packard 107

  • Sanjay's pre-final office hours: Friday Jun 6, 2-3:30

  • Samuel's pre-final office hours: Friday Jun 6, 8:30pm-10pm in Huang 219