AlphaZero Implementation

Environment Setup

Create a new conda environment:

conda create -n alphazero python=3.7
conda activate alphazero

Clone and setup repo

git clone https://github.com/mtb2718/alphazero.git
cd alphazero
pip install -e .

Install PyTorch

See PyTorch website for latest & most appropriate command for your setup.

On a laptop with no GPU, for instance, you may try the following command:

conda install pytorch torchvision cpuonly -c pytorch

This setup is only practical for training and experimenting in toy-problem settings, like tic-tac-toe.

Install other python dependencies via pip

pip install -r requirements.txt

Install Redis

Redis is an in-memory datastore used to sync the latest models during training with distributed self-play workers. Note that this dependency is not necessary for single-process training in toy-problem settings. (However, it is still currently required. TODO: make Redis dependency optional.)

Build from source and install (recommended):

wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make

You may now launch the src/redis-server binary in another tab before starting training, or alternatively follow the instructions in the Redis quickstart guide to finish by installing and daemonizing the server (recommended).

Install Connect4 solver (required for Connect4 evaluation)

Tic-tac-toe and Connect4 are strongly solved games, meaning that for any board position the best possible move(s) and the relative strength of each possible move are known a priori. As such, these games make for especially enlightening testbeds, as we can accurately evaluate both the predictions from our neural network models and the quality of the training targets produced by self-play. Solving tic-tac-toe is trivial, included in tictactoe.py in < 50 lines of code. In Connect4, quite a bit more effort is required to solve positions efficiently, so we rely on an existing open-source solver by Pascal Pons.

Clone and build the Connect4 solver.

git clone https://github.com/PascalPons/connect4.git
cd connect4
make c4solver

The solver optionally makes use of a "book" of precomputed solutions, which decreases solve times by orders of magnitude. Download the book of precomputed solutions.

Our ConnectFour class's solve() method assumes both the solver and book are "installed" in your system:

sudo cp c4solver /usr/local/bin/
sudo cp 7x6.book /usr/local/bin/

Run tests

Finally, confirm that everything is working as expected by running the included unit tests:

pytest