Project for the course Capita Selecta Computer Science: Artificial Intelligence at KU Leuven
A presentation on the topic "Human-level control through deep reinforcement learning". For our demo, we implemented an agent that is able to learn to play a variety of (simple) games using deep reinforcement learning.
- Intro to the problem this project tries to solve
- A refresher on reinforcement learning
- An introduction to convolutional neural networks
- Deep reinforcement learning
- Demo!
- Curious Reinforcment Learning, a short section on what may come after deep reinforcement learning.
The implementation of an agent that can successfully learn to play games can be found in src/main.py.
Our implementation closely follows Deepmind's Deep Q-Network.
The deep neural network takes as input 4 video frames (grayscale with resolution 84x84) and returns probabilities for next actions to take. It consists of 3 convolutional layers and 2 fully connected layers with ReLUs in between. The network uses Experience Replay and a target network as described in Deepmind's DQN paper.
For the agent to learn to play games, we needed a virtual environment.
- Can host a variety of games
- Get observations from environment
- Make actions in environment
- Receive reward for action from environment
- Tracks to drive on
The environment we ended up using OpenAI's Gym.
We originally started off with Udacity's Behavioral cloning project to create an agent for self driving cars. That path ended up being unfruitful, as we were not able to synchronise the data of the simulator with the separate neural network program correctly.
We then opted for a simpler approach in order to stay within the allocated time for this project. OpenAI's Gym library allowed us to write a working program in much less time. It also gave us the possibility to try out our network on a variety of different games.
This project relies on the following python dependencies:
numpy pytorch gym gym[atari]
The agent can be trained as follows
python3 main.py
The agent will start learning and will output the achieved score and save its network's weights every 10 episodes.
The agent can afterwards be loaded and be used to play the game with
python3 play.py
- OpenAI Gym - Enviroments to interact with games
- OpenAI Universe - Training and evaluating AI agents
- PyTorch - Deep neural networks engine with GPU acceleration
- Behavioral Cloning Project - Udacity's car simulation environment which allows for neural networks to learn to drive cars autonomously around tracks.
- Thiery Deruyterre - ThierryDeruyttere
- Armin Halilovic – arminnh
Distributed under the MIT license. See LICENSE for more information.
- Human-level control through deep reinforcement learning
- A Brief Survey of Deep Reinforcement Learning
- Dueling Network Architectures for Deep Reinforcement Learning
- Deep Recurrent Q-Learning for Partially Observable MDPs
- Curiosity-driven Exploration by Self-supervised Prediction
- A REINFORCEMENT LEARNING ALGORITHM FOR NEURAL NETWORKS WITH INCREMENTAL LEARNING ABILITY
- DEEP REINFORCEMENT LEARNING: AN OVERVIEW
- Reinforcement Learning for Robots Using Neural Networks
- Human-level control through deep reinforcement learning slides
- NEURAL NETWORKS AND REINFORCEMENT LEARNING
- MIT 6.S094: Deep Learning for Self-Driving Cars
- OpenAI Baselines: DQN
- Learning Diverse Skills via Maximum Entropy Deep Reinforcement Learning
- Curiosity Driven Exploration by Self-Supervised Prediction
- Clever Machines Learn How to Be Curious
- A solution for Udacity's Self-Driving Car project
- Self-Driving Truck in Euro Truck Simulator 2, trained via Reinforcement Learning
- Competitive Self-Play
- Stock market environment using OpenGym with Deep Q-learning and Policy Gradient
- End to End Learning for Self-Driving Cars
- ConvNetJS Deep Q Learning Demo
- ConvNetJS: Deep Learning in your browser
- A Beginner's Guide To Understanding Convolutional Neural Networks
- Vanilla DQN, Double DQN, and Dueling DQN in PyTorch