/reinforcement_learning_resources

Personal Deep Reinforcement Learning class notes

Reinforcement Learning resources

Keywords: Deep Reinforcement Learning, UC Berkeley

This repository contains some personal Deep Reinforcement Learning class notes in .pdf format.

Table of contents

  1. Class notes content
    1.1 CS285 (UC Berkeley) - Deep Reinforcement Learning by Sergey Levine
  2. Resources

1. Class notes content

1.1 CS285 (UC Berkeley) - Deep Reinforcement Learning by Sergey Levine

These notes summarize the main Reinforcement Learning algorithms, both in theory and in practice with some tips & hacks for efficient implementation.

1. Introduction

WIP

2. Supervised Learning of behaviors

  1. Goal
  2. Algorithms
    2.1 DAgger: Dataset Aggregation
  3. Tips & hacks

3. Introduction to Reinforcement Learning

  1. Goal
  2. Algorithms
    2.1 Global structure
    2.2 Exemples

4. Policy Gradients

  1. Algorithms
    1.1. REINFORCE
  2. Tips & hacks

5. Actor-Critic algorithms

  1. Algorithms
    1.1. Batch Actor-Critic
    1.2. Online Actor-Critic
  2. Tips & hacks

6. Value function methods

  1. Algorithms
    1.1. Policy iteration
    1.2. Policy iteration with Dynamic programming
    1.3. Value iteration
    1.4. Fitted Value iteration
    1.5. Fitted Q-iteration
    1.6. Online Q-iteration
  2. Tips & hacks

7. Deep Reinforcement Learning with Q-functions

  1. Algorithms
    1.1. Q-learning with replay buffer
    1.2. Q-learning with replay buffer and target network
    1.3. DQN: classic Deep Q-learning
    1.4. DDPG: Q-learning for continuous actions
  2. Tips & hacks

8. Advanced Policy Gradients

WIP

9. Model-based planning

WIP

10. Model-based Reinforcement Learning

  1. Algorithms
    1.1. Model-based Reinforcement Learning version 0.5
    1.2. Model-based Reinforcement Learning version 1.0
    1.3. Model-based Reinforcement Learning version 1.5
    1.4. Model-based Reinforcement Learning with latent space models
  2. Tips & hacks

11. Model-based Policy Learning

  1. Algorithms
    1.1. Model-based Reinforcement Learning version 2.0
    1.2. DYNA: online Q-learning model-free Reinforcement Learning with a model
    1.3. General DYNA-style model-based Reinforcement Learning
    1.4. MBA: Model-based Acceleration – MVE: Model-based Value Ex- pansion – MBPO: Model-based Policy Optimization
    1.5. Divide and Conquer Reinforcement Learning
  2. Tips & hacks

12. Variational Inference & Generative models

WIP

13. Control as inference

  1. Algorithms
    1.1 Soft Q-learning

14. Inverse Reinforcement Learning

  1. Algorithms
    1.1. Maximum Entropy Inverse Reinforcement Learning
  2. Tips & hacks

15. Transfer & Multi-task Learning

  1. Tips & hacks

16. Distributed Reinforcement Learning

WIP

17. Exploration

  1. Algorithms
    1.1. Pre-train & finetune
  2. Tips & hacks

18. Meta-learning

WIP

19. Information theory

WIP

2. Resources

These notes were widely inpspired by: