This course covers the fundamental techniques in applied Reinforcement Learning (RL).
Reinforcement Learning is one the most successful subdomains within Artificial Intelligence (AI). The core idea in Reinforcement Learning is agents learn to which actions are successful in a given environment.
Reinforcement Learning extends the usefulness of Machine Learning into new domains, such as video games, board games, robotics, and using human feedback to improve machine learning models (i.e., training GPTChat). The course will focus on contemporary, practical applications of Reinforcement Learning, such as multi-arm bandits, q-learning, and Deep Reinforcement Learning.
Instructor: Brian Spiering
Website: github.com/brianspiering/
rl-course
By the end of the course, you should be able to:
- Apply Reinforcement Learning techniques to solve complex, applied problems.
- Define and identify examples of common Reinforcement Learning terms (e.g., agent, environment, state, and rewards).
- Implement common Reinforcement Learning techniques (e.g., multi-arm bandits and q-learning) from scratch.
- Create end-to-end Deep Reinforcement Learning applications.
- Working knowledge of probability, statistics, machine learning, and deep learning.
- Intermediate level of Python (e.g., ability to create to classes).
- Welcome
- Intro to RL
- Markov Decision Processes (MDP)
- Policy
- Value Iteration
- Multi-armed Bandits for AB testing
- Contextual Bandits for Recommendation Systems
- Dynamic Programming
- Monte Carlo
- Temporal Difference Learning
- Q-Learning
- Value Function Approximation
- Deep Reinforcement Learning (DRL)
- Deep Q-learning
- Deep Reinforcement Learning Applications
- Proximal Policy Optimization (PPO)
- Alpha Go
- Future of RL
- Student's Choice
The Deep Learning section will be in Keras.
- Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto. The standard textbook for RL pdf
- Algorithms for Reinforcement Learning by Csaba Szepesvari. A delightful walk-through of the most common RL algorithms pdf
- Artificial Intelligence: A Modern Approach 4ᵗʰ Edition by Stewart Russell and Peter Norvig. The standard textbook for AI. website
| Item | Weight |
|---|---|
| Prework Check | 10% |
| Participation | 10% |
| Exam | 20% |
| Labs | 30% |
| Final Project | 30% |
| Total | 100% |
Each item's contribution is capped its respective percentage. The total course percentage is capped at 100%.
Currently, there is no extra credit. If there is any extra credit, it entirely at the discretion of the instructor.
I try to create an active learning environment in my classroom, which is incentivized with the Participation grade. Attendance is mandatory, you can't participate if you don't attend. It is the responsibility of the student to attend all classes. If you have to miss class, due to sickness, job related activities, or other circumstances, please notify your instructor by Slack in advance.
Tardiness negatively impacts an active learning environment, thus will impact your participation grade.
You must show up to each session prepared. Each person is important to the dynamic of the class, and therefore students are required to participate in class activities. Expect to be "cold called". I call on students at random not to put you on the spot but to keep you engaged in the material at all times.
I expect you to be fully present and engaged at all times.
I except you follow the etiquette guidelines throughout the entire course. This is your warning. Every violation will result in a loss of participation points, negatively impacting your total grade.
There will be a required prework before each class. The prework will check that you have watch the videos, read the reading, and are prepared for the in-class activities. The prework check also allows be to get a sense for the current understanding of the class. I know which material I can skim/skip and which material to over in greater detail.
The prework will be online in Canvas.
The single exam will be a take-home exam.
The labs will be hands-on activities. The focus will be implementing algorithms from scratch or applying common libraries.
- Tower of Hanoi
- Grid World
- Pyramid Escape
- Tic, Tac, Toe
- Basket Catch
- Cart Pole
In lieu of a Final Exam, there will be a Final Project. The Final Project will be implement a Reinforcement Learning solution for a novel task. By novel, I mean a task that has no existing Reinforcement Learning solutions. The task could be a game, a dataset, or a business problem.
Details and due dates are in Final Project Folder.