These lecture slides will be updated frequently, both before and after the lecture is covered in class. We will also be adding more lectures, so the numbers will change as well.

Introduction

Shortest path example

Probability and Monte Carlo

Markov chains

Epidemic example

Hitting times

Structure of Markov chains

Value

Cost and reward

Dynamic pricing example

Markov decision processes

The Bellman-Ford algorithm

Dynamic programming

Code for dynamic programming

Dynamic programming proof

More information patterns

Linear quadratic stochastic control

Linear quadratic regulator

Linear quadratic trading example

Approximate dynamic programming

Model predictive control

Shortest paths

Informed search

Risk averse control

Linear exponential quadratic regulator

Hidden Markov models