CS-456-Artificial-Neural-Networks-Project

Q-Learning and Deep Q-Learning to train artificial agents that can play the famous game of Nim.

2022 miniproject of the Artificial Neural Networks course at EPFL (code CS-456), given by the very well-known Wulfram Gerstner and Johanni Brea.

File architecture

q_learning.ipynb : Jupyter Notebook containing the Q-Learning part of the project (questions 1 to 10)
q_learning_results.pkl : pickle file to reload all results of the previous notebook without having to re-run the main game function (q_learning)
deep_q_learning.ipynb : Jupyter Notebook containing the Deep Q-Learning part of the project, as well as the comparison between Q-Learning and Deep Q-Learning (questions 11 to 21)
deep_q_learning_results.pkl : pickle file to reload all results of the previous notebook without having to re-run the main game function (deep_q_learning)
nim_env.py : Python file defining the environment of the game of Nim

Hardware

GPU: GeForce RTX 3070
CPU: Ryzen 7 3700x
RAM: 16 GB

Required libraries

torch
tqdm
random
numpy
collections
plotly.express
pandas
multiprocessing
joblib
pickle
time

How to run the code ?

First, clone the repository. Then, you can run each of the cells of both notebooks to test any game configuration you like. The notebooks follow the order of the questions in MP_Nim.pdf.

Implementation details

In each approach, it is possible to assess the performance of our learning agent against optimal and random players by setting the test variable to True. It is also possible to make our agent play against itself, i.e. by changing the instance of the adversary player with the boolean self_learning.

Authors

Elia Fantini
Félix Klein