/Awesome-LLMOps

🎉 An awesome & curated list of best LLMOps tools.

Awesome-LLMOps Awesome

🎉 An awesome & curated list of best LLMOps tools. But more about LLMOps.

Table of Contents

LLMOps

Name Stars Release Contributors About
Dify Stars Release Contributors One API for plugins and datasets, one interface for prompt engineering and visual operation, all for creating powerful AI applications
FastChat Stars Release Contributors An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Flowise Stars Release Contributors Drag & drop UI to build your customized LLM flow
LangChain Stars Release Contributors ⚡ Building applications with LLMs through composability ⚡
LiteLLM Stars Release Contributors lightweight package to simplify LLM API calls - Azure, OpenAI, Cohere, Anthropic, Replicate. Manages input/output translation
LLaMa-Factory Stars Release Contributors Easy-to-use LLM fine-tuning framework (LLaMA, BLOOM, Mistral, Baichuan, Qwen, ChatGLM)
LlamaIndex Stars Release Contributors LlamaIndex (GPT Index) is a data framework for your LLM applications
Ludwig Stars Release Contributors Low-code framework for building custom LLMs, neural networks, and other AI models
SkyPilot Stars Release Contributors SkyPilot: Run LLMs, AI, and Batch jobs on any cloud. Get maximum savings, highest GPU availability, and managed execution—all with a simple interface.

MLOps

Name Stars Release Contributors About
Flyte Stars Release Contributors Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
Kubeflow Stars Release Contributors Machine Learning Toolkit for Kubernetes
Metaflow Stars Release Contributors 🚀 Build and manage real-life data science projects with ease!
MLflow Stars Release Contributors Open source platform for the machine learning lifecycle
ZenML Stars Release Contributors ZenML 🙏: Build portable, production-ready MLOps pipelines. https://zenml.io.

Training

Name Stars Release Contributors About
ColossalAI Stars Release Contributors Making large AI models cheaper, faster and more accessible

Agent

Name Stars Release Contributors About
AutoGPT Stars Release Contributors An experimental open-source attempt to make GPT-4 fully autonomous.
MetaGPT Stars Release Contributors 🌟 The Multi-Agent Framework: Given one line Requirement, return PRD, Design, Tasks, Repo
XAgent Stars Release Contributors An Autonomous LLM Agent for Complex Task Solving

Inference

Name Stars Release Contributors About
DeepSpeed-MII Stars Release Contributors MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
OpenLLM Stars Release Contributors Operating LLMs in production
RayServe Stars Release Contributors Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Text Generation Inference Stars Release Contributors Large Language Model Text Generation Inference
vLLM Stars Release Contributors A high-throughput and memory-efficient inference and serving engine for LLMs

Evaluation

Name Stars Release Contributors About
AgentBench Stars Release Contributors A Comprehensive Benchmark to Evaluate LLMs as Agents

Database

Name Stars Release Contributors About
deeplake Stars Release Contributors Database for AI. Store Vectors, Images, Texts, Videos, etc. Use with LLMs/LangChain. Store, query, version, & visualize any AI data. Stream data in real-time to PyTorch/TensorFlow. https://activeloop.ai

Observation

Name Stars Release Contributors About
OpenLLMetry Stars Release Contributors Open-source observability for your LLM application, based on OpenTelemetry
phoenix Stars Release Contributors ML Observability in a Notebook - Uncover Insights, Surface Problems, Monitor, and Fine Tune your Generative LLM, CV and Tabular Models