/databricks-ml-examples

Primary LanguagePythonApache License 2.0Apache-2.0

databricks-ml-examples

databricks/databricks-ml-examples is a repository to show machine learning examples on Databricks platforms.

Currently this repository contains:

  • llm-models/: Example notebooks to use different State of the art (SOTA) models on Databricks.

SOTA LLM examples

Databricks works with thousands of customers to build generative AI applications. While you can use Databricks to work with any generative AI model, including commercial and research, the table below lists our current model recommendations for popular use cases. Note: The table only lists open source models that are for free commercial use.

Use case Quality-optimized Balanced Speed-optimized
Text generation following instructions MPT-30B-Instruct

Llama-2-70b-chat-hf
mistral-7b

MPT-7B-Instruct
MPT-7B-8k-Instruct

Llama-2-7b-chat-hf
Llama-2-13b-chat-hf
Text embeddings (English only) bge-large-en-v1.5(0.3B)
e5-large-v2 (0.3B)
bge-base-en-v1.5 (0.1B)
e5-base-v2 (0.1B)
Transcription (speech to text) whisper-large-v2(1.6B)
whisper-medium (0.8B)
Image generation stable-diffusion-xl
Code generation CodeLlama-34b-hf
CodeLlama-34b-Instruct-hf
CodeLlama-34b-Python-hf (Python optimized)
CodeLlama-13b-hf
CodeLlama-13b-Instruct-hf
CodeLlama-13b-Python-hf (Python optimized)
CodeLlama-7b-hf
CodeLlama-7b-Instruct-hf
CodeLlama-7b-Python-hf (Python optimized)

Other examples: