for a better guide than mine, click here
This repo is meant as a guide on how Llama3's architecture works in the same vein of Andrej Karpathy's minGPT (hint: it's basically the same as Llama2). Over in the colab notebook I'll hold your hand through every single operation performed in Llama 3, and in 'model.py' you can check out what that code looks like once it's turned into actual pytorch nn.Module objects. 'training.ipynb' and 'inference.ipynb' are what they sound like. there are 3 different models and 4 different tokenizers over in 'models/' and 'tokenizers/'. The only requirement you'll need to install in order for everything to run that doesn't come with python by default is pytorch. Check out the accompanying youtube video!
