Ashwin Rao

Adjunct Professor, ICME

Stanford University

Office: Huang Engineering Center, ICME Mezannine Level, Room M05

Email: ashwin.rao@stanford.edu

My academic background is in Algorithms Theory and Abstract Algebra.
My current academic interests lie in the broad space of A.I. for Sequential Decisioning under Uncertainty.
I am particularly interested in Deep Reinforcement Learning applied to Financial Markets and to Retail Businesses.
More details on my background and work are described on my LinkedIn page.

My RL-for-Finance book (with Tikhon Jelvis)

Book Title: Foundations of Reinforcement Learning with Applications in Finance

Publisher: Routledge and CRC Press (Taylor & Francis Group)

• Order the Published Book from Amazon
• Published Book Errata
• Download the latest PDF version of the book (this has updated contents, including fixes to the published book errors)
• Public-Access Git Repo for the code in the book
• Stanford CME 241 Home Page (I cover most of the book in this course, and the course home page has the lectures slides, assignments and exams)

RL-Lab Research Projects

Applied RL research projects I am curently working on with my students:

• Joint Optimization of Inventory and Pricing in a Retail Business
• Optimal Exercise of American Options (using LSPI and DQL)
• Personal Finance Optimization, including Retirement Planning

Current Teaching

• CME 241: Foundations of Reinforcement Learning with Applications in Finance (I teach this course every Winter Quarter)

An Incomplete Collection of my Past Research & Teaching

AI

• RL for Finance book
• RL for Finance course materials
• Function Approximations as Affine Spaces (colab)
• Adaptive Multistage Sampling Algorithms (origins of MCTS)
• Evolutionary Strategies as an alternative to RL
• Understanding Dynamic Programming through Bellman Operators
• AI for Dynamic Decisioning under Uncertainty
• Value Function Geometry and Gradient TD
• Policy Gradient Theorem and Algorithms
• Backpropagation
• Bias-Variance Tradeoff
• Attention is all you need (colab)
• Introduction to SparseMax (colab)

Mathematics

• Category Theory
• Abstract Algebra
• The Algebra of Random Number Generation
• Stochastic Calculus: Quick Intro (Colab), Overview (Slides), Tutorial (Slides)
• Riemann Hypothesis (colab)
• Moment Generating Functions
• Bayesian Trap (colab)
• 2025 (colab)
• Vampire Problem (colab)
• Quiz Game (colab)

Finance

• Continuous-Time Pricing Equations for Agency Mortgage-Backed Securities
• Towards Improved Pricing and Hedging of Agency Mortgage-Backed Securities
• Fundamental Theorems of Asset Pricing
• Portfolio Theory
• The Newsvendor meets the Options Trader
• Risk-Aversion through Utility Theory
• Silicon Valley Bank, Inverted Curve and Bond Risk (Story + code in colab)
• Price-Entry Spread as a tool to balance risk and reward (colab code)
• RL for Optimal Market Making
• Real-World Derivatives Hedging with Deep RL
• RL for Optimal Trade Order Execution
• RL for Pricing American Options
• HJB Equation and Merton’s Portfolio Problem
• Retirement Planning as a Linear Program (colab code)

Programming

• My github (several repos)
• Book github repo (joint work with Tikhon Jelvis)
• Programming and Design (Chapter 2 of RL Book covering Python tools)
• Abstracting over Computation (from Stanford summer workshop)
• Abstracting over Data (from Stanford summer workshop)

Supply-Chain

• Omni-channel Newsvendor
• Economic Order Quantity (EOQ) under Spoilage/Expiration
• The Newsvendor meets the Options Trader
• Markov Decision Process for Under/Over-Capacity Optimization
• Principles of Mathematical Economics applied to a Physical Stores Business
• Risk Pooling Sensitivity to Correlation
• Constrained Dynamic Program for Backroom Minimization Ensuring Adequate Shelf Inventory
• Lead Time and Ordering Frequency Analysis for Purchasing
• Joint Policy Optimization for n Stores and 1 DC
• ML Applications for Demand Forecasting and Inventory Replenishment
• Dynamic Programming Solution for Joint Inventory Control of 2 stores (colab code)
• Q-Learning (RL) algorithm for Store Inventory Control with Shrinkage (colab code)
• Joint Optimal Control of Inventory and Price (colab code)

Pricing, Sales & Marketing

• Optimal Pricing under Elasticity of Demand (colab code)
• Optimal Sale Price Maximizing Expected Sales Commission (colab code)
• Incrementality in Marketing
• Clearance Pricing (colab code)
• Analytic Forms for Discrete Time Clearance Price Optimization
• Value Gap Due to Price Optimization Model Error

Sports

• RL for Optimal Cricket Batting Risk Strategy (colab code)
• Optimal Control in a Race balancing Speed and Stamina (colab code)
• Fitness 101 - Mumbai