/llama-models

Utilities intended for use with Llama models.

Primary LanguagePythonOtherNOASSERTION

🤗 Models on Hugging Face  | Blog  | Website  | Get Started 

Llama Models

PyPI - Downloads Discord

Llama is an accessible, open large language model (LLM) designed for developers, researchers, and businesses to build, experiment, and responsibly scale their generative AI ideas. Part of a foundational system, it serves as a bedrock for innovation in the global community. A few key aspects:

  1. Open access: Easy accessibility to cutting-edge large language models, fostering collaboration and advancements among developers, researchers, and organizations
  2. Broad ecosystem: Llama models have been downloaded hundreds of millions of times, there are thousands of community projects built on Llama and platform support is broad from cloud providers to startups - the world is building with Llama!
  3. Trust & safety: Llama models are part of a comprehensive approach to trust and safety, releasing models and tools that are designed to enable community collaboration and encourage the standardization of the development and usage of trust and safety tools for generative AI

Our mission is to empower individuals and industry through this opportunity while fostering an environment of discovery and ethical AI advancements. The model weights are licensed for researchers and commercial entities, upholding the principles of openness.

Llama Models

Model Launch date Model sizes Context Length Tokenizer Acceptable use policy License Model Card
Llama 2 7/18/2023 7B, 13B, 70B 4K Sentencepiece Use Policy License Model Card
Llama 3 4/18/2024 8B, 70B 8K TikToken-based Use Policy License Model Card
Llama 3.1 7/23/2024 8B, 70B, 405B 128K TikToken-based Use Policy License Model Card

Download

To download the model weights and tokenizer:

  1. Visit the Meta Llama website.
  2. Read and accept the license.
  3. Once your request is approved you will receive a signed URL via email.
  4. Install the Llama CLI: pip install llama-toolchain
  5. Run llama model list to determine the model ID you wish to download
  6. Run: llama download --source meta --model-id CHOSEN_MODEL_ID
  7. Pass the URL provided when prompted to start the download.

Remember that the links expire after 24 hours and a certain amount of downloads. You can always re-request a link if you start seeing errors such as 403: Forbidden.

Download via HuggingFace

We also provide downloads on Hugging Face in both transformers and native llama3 formats. To gain access:

  1. Visit one of the repos (ex. meta-llama/Meta-Llama-3.1-8B-Instruct).
  2. Read and accept the license.
  3. Once your request is approved, you'll be granted access to all Llama 3.1 models as well as previous versions. Note that approvals may take up to one hour.

You can then download the models:

  • Via llama download --source huggingface --hf-token YOUR_ACCESS_TOKEN (create/view access tokens here)
  • Via the HuggingFace CLI (pip install huggingface-hub):
  • In the web browser by clicking on the "Files and versions" tab
huggingface-cli download meta-llama/Meta-Llama-3.1-8B-Instruct --include "original/*" --local-dir meta-llama/Meta-Llama-3.1-8B-Instruct

The original native weights are in the original/ subfolder (except for meta-llama/Meta-Llama-3.1-405B).

Using with transformers

  • To use with transformers, the following pipeline snippet will download and cache the weights:

    import transformers
import torch

model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct"

pipeline = transformers.pipeline(
  "text-generation",
  model="meta-llama/Meta-Llama-3.1-8B-Instruct",
  model_kwargs={"torch_dtype": torch.bfloat16},
  device="cuda",
)

Installations

You can install this repository as a package by just doing pip install llama-models

Responsible Use

Llama models are a new technology that carries potential risks with use. Testing conducted to date has not — and could not — cover all scenarios. To help developers address these risks, we have created the Responsible Use Guide.

Issues

Please report any software “bug” or other problems with the models through one of the following means:

Questions

For common questions, the FAQ can be found here, which will be updated over time as new questions arise.