/reinforcement-learning-an-introduction

Python code for Reinforcement Learning: An Introduction

Primary LanguagePython

Reinforcement Learning: An Introduction

Python code for Sutton & Barto's book Reinforcement Learning: An Introduction

Contents

Chapter 1

  1. Tic-Tac-Toe

Chapter 2

  1. Figure 2.2: Average performance of epsilon-greedy action-value methods on the 10-armed testbed
  2. Figure 2.3: Optimistic initial action-value estimates
  3. Figure 2.4: Average performance of UCB action selection on the 10-armed testbed
  4. Figure 2.5: Average performance of the gradient bandit algorithm

Chapter 3

  1. Figure 3.5: Grid example with random policy
  2. Figure 3.8: Optimal solutions to the gridworld example

Chapter 4

  1. Figure 4.1: Convergence of iterative policy evaluation on a small gridworld
  2. Figure 4.2: Jack’s car rental problem
  3. Figure 4.3: The solution to the gambler’s problem

Chapter 5

  1. Figure 5.1: Approximate state-value functions for the blackjack policy
  2. Figure 5.4: Weighted importance sampling
  3. Figure 5.5: Ordinary importance sampling with surprisingly unstable estimates

Chapter 6

  1. Figure 6.2: Random walk
  2. Figure 6.3: Batch updating
  3. Figure 6.4: Sarsa applied to windy grid world
  4. Figure 6.5: The cliff-walking task
  5. Figure 6.7: Interim and asymptotic performance of TD control methods
  6. Figure 6.8: Comparison of Q-learning and Double Q-learning

Chapter 7

  1. Figure 7.2: Performance of n-step TD methods on 19-state random walk