These lecture slides will be updated frequently, both before and after the lecture is covered in class. We will also be adding more lectures, so the numbers will change as well.

Introduction

Shortest path example

Probability and Monte Carlo

Markov chains

Epidemic example

Hitting times

Structure of Markov chains

Value

Cost and reward

Dynamic pricing example

Markov decision processes

The Bellman-Ford algorithm

Dynamic programming