codeninja/gpt-neo-fine-tuning-example

Fine-Tune EleutherAI GPT-Neo to Generate Netflix Movie Descriptions Using Hugginface And DeepSpeed

GPT-Neo-2.7B Fine-Tuning Example Using HuggingFace & DeepSpeed

Installation

cd venv/bin
./pip install -r ../../requirements.txt 
./pip install deepspeed==0.3.13

Example with GPT-Neo-2.7B with DeepSpeed
Example with GPT-Neo-1.3B without DeepSpeed
DeepSpeed configuration
Training and testing log

GPU VRAM load during training

Resilts