lucabeetz/transformer

Minimal GPT-2 implementation

transformer

Minimal GPT-2 implementation with byte-pair encoder.

Supports pretrained weights from HuggingFace for gpt2, gpt2-medium, gpt2-large, gpt2-xl.

Usage

python scripts/generate.py <prompt>

Links

• Andrej Karpathy's Min-GPT
• Original GPT-2 code by OpenAI

GPT-2 Quotes

• Machine learning is a way of doing things that is not as simple as it could be. But it's not the only way.