/ReinforcementLearningBookExamples

Example codes to implement the examples in Richard's book, Reinforcement Learning: An Introduction.

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

ReinforcementLearningBookExamples

Example codes to implement the examples in Richard's book, Reinforcement Learning: An Introduction.

Scripts

  1. 1TenArmedBandits.py: code scrip regarding to the k-armed bandit problem in Chapter 2.

    • Bandit class
    • Agent class
    • epsilon-greedy algorithm
    • optimistic initial values
    • ucb
    • gradient algorithm

  2. 2GridWorld_Ch3.py: code script regarding to example in Chapter 3

    • GridWorld
    • values estimated based on Bellman equation
    • values estimated based on Bellman optimal equation

  3. 3Carrental_Ch4.py: code script for car rental problem in Chapter4. However, there still exists bug in maybe get_expected_return function.

    • JackRentalCompany, class simulating the car rental company
    • policy_evaluate in Agent class
    • policy_improve in Agent class
    • debug!!!

  4. 6CliffWalk_Ch6.py: code script for handling cliff walking problem in Chapter 6 by SARSA and Q-LEANING.

    • CliffWalk, lass simulaitng the cliff walking game
    • Q_net, class to store q-values
    • train_sarsa, function to estimate q values by SARSA
    • train_q_learning, function to estimate q values by Q-learning

  5. 13CliffWalking_Ch13.py: code script for cliff walking problem in Chapter 13 with various methods.

    • CliffWalk, class simulaitng the cliff walking game
    • REINFORCE
    • REINFORCE with Baseline
    • Actor-Critic w/o eligibility trace

Todo:

  1. Car Rental in Chapter 4
  2. Various methods for solving Blackjack in Chapter 5
  3. Q-learning for Cliff Walking in Chapter 6
  4. Policy Gradient Methods for Cliff Walking problem in Chapter 13