/ChessGPT

Training NanoGPT-based language models specialized in chess gameplay to beat me at chess.

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

ChessGPT

This repo contains code for training your own NanoGPT-based models on chess data.

Getting Started

Follow these steps to set up and train your ChessGPT model:

Prerequisites

Ensure you have Python 3.7 or later installed.

Setup Instructions

  1. Clone the Repo: Begin by cloning this repository to your local machine using:

    git clone https://github.com/jmtzt/ChessGPT.git
cd ChessGPT

    Also make sure to install the pre-commit hooks by running the following command:

    pre-commit install

  2. Download the Dataset: Download the chess dataset from Hugging Face Datasets. Place the downloaded files in the data/lichess directory. For the initial experiments, I've used the lichess_6gb_blocks dataset.

  3. Install Dependencies: Install the required Python packages in your virtual env using:

    pip install -r requirements.txt

  4. Prepare the Data: Run the src/datamodules.py script to tokenize and preprocess the data:

    python src/datamodules.py

    This will save the preprocessed files in binary format in the data/lichess directory.

  5. Configure the Model: Adjust the model configuration in src/train.py to suit your requirements.

    The training script includes several configuration options you can adjust:

    • Model Parameters: Change the model's architecture, initialization, number of attention heads, embedding size, etc.
    • Data Handling: Configure how the data is loaded and processed.
    • Training Parameters: Set the number of epochs, batch size, learning rate as needed.

    Also make sure to either setup Weights & Biases or comment out the relevant lines in the training script.

  6. Train the Model: To start training the model, simply execute:

    python src/train.py

    This script initializes the model, sets up the data module, and begins the training process.

Acknowledgements