/MOOC-Udemy-Modern_Reinforcement_Learning_Deep_Q_Learning_in_PyTorch

Primary LanguagePython

Modern Reinforcement Learning: Deep Q Learning in PyTorch

Phil Tabor, Udemy

Syllabus

Section 1: Introduction

    1. What You Will Learn In This Course
    1. Required Background, software, and hardware
    1. How to Succeed in this Course

Section 2: Fundamentals of Reinforcement Learning

    1. Agents, Environments, and Actions
    1. Markov Decision Processes
    1. Value Functions, Action Value Functions, and the Bellman Equation
    1. Model Free vs. Model Based Learning
    1. The Explore-Exploit Dilemma
    1. Temporal Difference Learning

Section 3: Deep Learning Crash Course

    1. Dealing with Continuous State Spaces with Deep Neural Networks
    1. Naive Deep Q Learning in Code: Step 1 - Coding the Deep Q Network
    1. Naive Deep Q Learning in Code: Step 3 - Coding the Main Loop and Learning
    1. Naive Deep Q Learning in Code: Step 4 - Verifying the Functionality of Our Code
    1. Naive Deep Q Learning in Code: Step 5 - Analyzing Our Agent's Performance
    1. Dealing with Screen Images with Convolutional Neural Networks

Section 4: Human Level Control Through Deep Reinforcement Learning, From Paper to Code

    1. How to Read Deep Learning Papers
    1. Analyzing the Paper
    1. How to Modify the OpenAI Gym Atari Environments
    1. How to Preprocess the OpenAI Gym Atari Screen Images
    1. How to Stack the Preprocessed Atari Screen Images
    1. How to Combine All the Changes
    1. How to Add Reward Clipping, Fire First, and No Ops
    1. How to Code the Agent's Memory
    1. How to Code the Deep Q Network
    1. Coding the Deep Q Agent: Step 1 - Coding the Constructor
    1. Coding the Deep Q Agent: Step 2 - Epsilon-Greedy Action Selection
    1. Coding the Deep Q Agent: Step 3 - Memory, Model Saving and Network Copying
    1. Coding the Deep Q Agent: Step 4 - The Agent's Learn Function
    1. Coding the Deep Q Agent: Step 5 - The Main Loop and Analyzing the Performance

Section 5: Deep Reinforcement Learning with Double Q Learning

    1. Analyzing the Paper
    1. Coding the Double Q Learning Agent and Analyzing Performance

Section 6: Dueling Network Architectures for Deep Reinforcement Learning

    1. Analyzing the Paper
    1. Coding the Dueling Deep Q Network
    1. Coding the Dueling Deep Q Learning Agent and Analyzing Performance
    1. Coding the Dueling Double Deep Q Learning Agent and Analyzing Performance

Section 7: Improving On Our Solutions

    1. Implementing a Command Line Interface for Rapid Model Testing
    1. Consolidating Our Code Base for Maximum Extensability
    1. How to Test Our Agent and Watch it Play the Game in Real Time

Section 8: Conclusion

    1. Summarizing What We've Learned

Section 9: Bonus Lecture

    1. Bonus Video: Where to Go From Here

Software Requirements

  • Python 3.6

wget https://www.python.org/ftp/python/3.6.13/Python-3.6.13.tar.xz
tar -xvf Python-3.6.13.tar.xz && rm -v Python-3.6.13.tar.xz && cd Python-3.6.13
apt-get install build-essential sqlite3 libsqlite3-dev libbz2-dev zlib1g-dev
./configure && make && make altinstall

  • venv

mkdir venv && python3.6 -m venv venv/
source venv/bin/activate
deactivate

  • zlib1g-dev, cmake, ffmpeg

sudo apt-get install zlib1g-dev cmake ffmpeg

  • torch, torchvision, numpy, matplotlib, gym, box2d-py, atari-py, opencv-python

(venv) pip3 install torch torchvision numpy matplotlib gym box2d-py atari-py opencv-python