Overview

This is a collection of notes and examples on reinforcement learning. These are based on the second edition of Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Bartow.

Contents

The following notes and examples are contained in this repository:

Chapter 2 - Multi-arm Bandits

• Notes
• Epsilon Greedy
• Upper Confidence Bound
• Gradient Ascent

Chapter 3 - Finite Markov Decision Processes

• Notes
• Gridworld

Chapter 4 - Dynamic Programming

• Notes
• Gridworld
• Jack's Car Rental
• Gambler's Problem

Chapter 5 - Monte Carlo Methods

• Notes
• Blackjack
• Solving Blackjack
• Off-policy Estimation of a Blackjack State Value
• Infinite Variance
• Racetrack

Chapter 6 - Temporal-Difference Learning

• Notes
• Random Walk
• Random Walk under Batch Updating
• Windy Gridworld
• Cliff Walking