These lecture slides will be updated frequently, both before and after the lecture is covered in class. We will also be adding more lectures, so the numbers will change as well.
Introduction
Shortest path example
Probability and Monte Carlo
Markov chains
Epidemic example
Hitting times
Structure of Markov chains
Value
Cost and reward
Dynamic pricing example
Markov decision processes
The Bellman-Ford algorithm
Dynamic programming
Code for dynamic programming
Dynamic programming proof
More information patterns
Linear quadratic stochastic control
Linear quadratic regulator
Linear quadratic trading example
Approximate dynamic programming
Model predictive control
Shortest paths
Informed search
Risk averse control
Linear exponential quadratic regulator
Hidden Markov models