This repository hosts the example code for loading and executing the Grok-1 model, which is available with open weights. The Grok-1 model is a state-of-the-art machine learning model designed for a wide range of applications.
To utilize the Grok-1 model, please follow these instructions:
-
Download the Checkpoint: First, ensure you have downloaded the checkpoint. Place the
ckpt-0directory within thecheckpointfolder. -
Install Dependencies: Install the required Python packages by running the following command in your terminal:
pip install -r requirements.txt
-
Execute the Model: You can run the model using the provided script. Execute the following command:
python run.py
This script will load the checkpoint and sample from the model using a test input.
Due to the extensive size of the Grok-1 model, which consists of 314 billion parameters, it is imperative to use a machine equipped with sufficient GPU memory to run the example code effectively.
It is important to note that the implementation of the Mixture of Experts (MoE) layer in this repository prioritizes simplicity over efficiency. This decision was made to eliminate the necessity for custom kernels, thereby facilitating the validation of the model's accuracy.
The Grok-1 model weights can be acquired using a torrent client with the following magnet link:
magnet:?xt=urn:btih:5f96d43576e3d386c9ba65b883210a393b68210e&tr=https%3A%2F%2Facademictorrents.com%2Fannounce.php&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce
Thanks goes to these wonderful contributors (emoji key following all-contributors specification):
 ibab 💻 📖 🚇 🚧 🤔 👀 🔧 |
 TobyPDE 💻 👀 🐛 🤔 🚧 |
Contributions from those not on the research team are welcome.
The code and associated Grok-1 weights in this release are licensed under the Apache 2.0 license. The license only applies to the source files in this repository and the model weights of Grok-1.