/Move_37_Syllabus

This is the syllabus for "Move 37", Siraj Raval's new course at School of AI

Move 37

Course Objective

This is the syllabus for "Move 37", Siraj Raval's free reinforcement learning course, as part of School of AI. This course can be taken for free on Youtube in the form of a playlist, or at School of AI for a more immersive learning experience. Reinforcement learning is driving some of the latest advances in AI, from DeepMind's AlphaGo to OpenAI's DOTA bots. Although these AIs are designed for video games, reinforcement learning is a powerful branch of AI that can be applied to endless applications in the real world. In this course, we'll cover various RL techniques in order of increasing complexity, applying them to both simulated and real world problems. Students will develop an intuition around when to use certain RL algorithms and by the end of the course will have the practical skills necessary to apply RL to a problem they are passionate about to make a positive impact in the world.

Prerequisites

  • Understand Basic Python Syntax
  • Understand how the Backpropagation algorithm works.

Components

  • Midterm Project
  • Final Project
  • Educational Videos
  • Quizzes
  • Reading Assignments
  • Coding Assignments
  • Interviews
  • Group Discussion in Slack

Course Length

  • 10 Weeks
  • 10-15 hours of dedicated study per week
  • Starts September 10 at 12 PM PST

Tools Used

  • Pytorch & Tensorflow (Deep Learning Libraries in Python)
  • OpenAI Gym (Reinforcement Learning Library)
  • Google Colab (for free GPUs, no need to install/configure dependencies)

Week 1 - Introduction

  • Introduction
  • Sensor Networks
  • Supply Chain Management
  • Energy Efficiency
Topics Covered
Markov Decision Processes, Policy Functions, Value Functions, and the Bellman Equation

Week 2 - Dynamic Programming

  • Route Planning
  • Options Pricing
  • Scheduling
  • Operating Systems
Topics Covered
Iterative Policy Evaluation, Policy improvement, Policy iteration, Value iteration

Week 3 - Monte Carlo Methods

  • Interview #1
  • Medical Diagnosis
  • Network Routing Optimization
  • Physics Research
Topics Covered
Monte Carlo prediction, Monte Carlo control, Greedy & Epsilon-Greedy Policies , Exploration vs Exploitation Dilemma

Week 4 - Model-Free Learning

  • Delivery Management
  • Automated Trading
  • Backgammon
  • Dopamine in Neuroscience
Topics Covered
Temporal Difference Learning, SARSA, Q-Learning, Model vs Model Free Intuition

Week 5 - Reinforcement Learning in Continuous Spaces

  • Self Driving Cars
  • Delivery Drones
  • Rescue Robots
  • Assembly Robots
Topics Covered
Control Theory, Imitation Learning, The Hamilton-Jacobi-Bellman Equation, Kalman Filters

Midterm Project

  • Train a bipedal humanoid robot to walk in simulation!

Week 6 - Deep Reinforcement Learning

  • Traffic Optimization
  • Gaming
  • Meta Learning
  • Interview #2
Topics Covered
DQN + Double DQN Networks, Dueling DQN, Prioritized Replay, Value-based Methods for Robotics

Week 7 - Policy Based Methods

  • Web System Configuration
  • Text Summarization
  • AI Assisted Design
  • Portfolio Optimization
Topics Covered
Evolutionary Algorithms, Stochastic Policy Search, Policy Gradients, REINFORCE

Week 8 - Policy Gradient Methods

  • Dialogue Systems
  • Photo Editing
  • Language Translation
  • Tutoring Systems
Topics Covered
Evolved Policy Gradients, Generalized Advantage Estimation (GAE), Trust Region Policy Optimization, Proximal Policy Optimization (PPO)

Week 9 - Actor Critic Methods

  • Advanced Trading Techniques
  • Human-Machine Cooperation
  • Insurance Cost Analysis
  • Interview #3
Topics Covered
Actor Critic Algorithms, Asynchronous Advantage Actor Critic, Deep Deterministic Policy Gradients (DDPG), Bayesian Actor-Critic

Week 10 - Multi Agent Reinforcement Learning

  • Move 37
  • Transportation Networks
  • Decentralized Autonomous Organizations
  • The Future of AI
Topics Covered
Cooperation, Competiton, Parallelism, Inverse Reinforcement Learning

Final Project

  • Develop a multi-agent network to solve a real world problem!