/openllm-models

Primary LanguagePython

The default model repository of openllm

This repo (on main branch) is already included by openllm by default.

If you want more up-to-date untested models, please add our nightly branch.

openllm repo add nightly https://github.com/bentoml/openllm-models@nightly

Supported Models

Table of Contents

Llama-3.1

Model Version Huggingface Link
llama3.1 405b-instruct-awq-4bit-df2a HF Link
llama3.1 70b-instruct-awq-4bit-2988 HF Link
llama3.1 70b-instruct-fp16-ace8 HF Link
llama3.1 8b-instruct-awq-4bit-fe8c HF Link
llama3.1 8b-instruct-fp16-2f36 HF Link

Llama-3

Model Version Huggingface Link
llama3 70b-instruct-awq-4bit-aebb HF Link
llama3 70b-instruct-fp16-1315 HF Link
llama3 8b-instruct-awq-4bit-3f34 HF Link
llama3 8b-instruct-fp16-8f83 HF Link

Phi-3

Model Version Huggingface Link
phi3 3.8b-instruct-fp16-166c HF Link
phi3 3.8b-instruct-ggml-q4-76aa HF Link

Mistral

Model Version Huggingface Link
mistral 24b-instruct-nemo-ec54 HF Link
mistral 7b-instruct-awq-4bit-01cd HF Link
mistral 7b-instruct-fp16-e1cd HF Link

Gemma-2

Model Version Huggingface Link
gemma2 27b-instruct-fp16-6b83 HF Link
gemma2 9b-instruct-fp16-6e86 HF Link

Qwen-2

Model Version Huggingface Link
qwen2 0.5b-instruct-fp16-33df HF Link
qwen2 1.5b-instruct-fp16-7cda HF Link
qwen2 57b-a14b-instruct-fp16-365f HF Link
qwen2 72b-instruct-awq-4bit-33fa HF Link
qwen2 72b-instruct-fp16-8cb4 HF Link
qwen2 7b-instruct-awq-4bit-14aa HF Link
qwen2 7b-instruct-fp16-bbf2 HF Link

Gemma

Model Version Huggingface Link
gemma 2b-instruct-fp16-6ee7 HF Link
gemma 7b-instruct-awq-4bit-df0b HF Link
gemma 7b-instruct-fp16-2297 HF Link

Llama-2

Model Version Huggingface Link
llama2 13b-chat-fp16-ef61 HF Link
llama2 70b-chat-fp16-16a0 HF Link
llama2 7b-chat-awq-4bit-4f93 HF Link
llama2 7b-chat-fp16-21b9 HF Link

Mixtral

Model Version Huggingface Link
mixtral 8x7b-instruct-v0.1-awq-4bit-06fd HF Link
mixtral 8x7b-instruct-v0.1-fp16-e289 HF Link

Mistral-Large

Model Version Huggingface Link
mistral-large 123b-instruct-awq-4bit-1d37 HF Link
mistral-large 123b-instruct-fp16-5c96 HF Link

Codestral

Model Version Huggingface Link
codestral 22b-v0.1-fp16-b677 HF Link