daypu/simple-gpt

Python

simple-gpt

Intro

This project implements a simple chat program based on the Transformer architecture, trained using the openwebtext dataset.

Run

Use the following code to install the required dependencies.

pip install -r requirements.txt

Run data-process.py, the program will automatically download the dataset and divide it into a training set and a validation set.
Run train.py to train your own model
Run chat.py to start the chat using the model you trained.

References

https://github.com/Infatoshi/fcc-intro-to-llms/tree/main