Awesome-LLMOps
๐ An awesome & curated list of best LLMOps tools. But more about LLMOps.
Name
Stars
Release
Contributors
About
BentoML
Build Production-Grade AI Applications
Dify
One API for plugins and datasets, one interface for prompt engineering and visual operation, all for creating powerful AI applications
FastChat
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Flowise
Drag & drop UI to build your customized LLM flow
LangChain
โก Building applications with LLMs through composability โก
LiteLLM
lightweight package to simplify LLM API calls - Azure, OpenAI, Cohere, Anthropic, Replicate. Manages input/output translation
LLaMa-Factory
Easy-to-use LLM fine-tuning framework (LLaMA, BLOOM, Mistral, Baichuan, Qwen, ChatGLM)
LlamaIndex
LlamaIndex (GPT Index) is a data framework for your LLM applications
Ludwig
Low-code framework for building custom LLMs, neural networks, and other AI models
SkyPilot
SkyPilot: Run LLMs, AI, and Batch jobs on any cloud. Get maximum savings, highest GPU availability, and managed executionโall with a simple interface.
Name
Stars
Release
Contributors
About
Flyte
Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
Kubeflow
Machine Learning Toolkit for Kubernetes
Metaflow
๐ Build and manage real-life data science projects with ease!
MLflow
Open source platform for the machine learning lifecycle
ZenML
ZenML ๐: Build portable, production-ready MLOps pipelines. https://zenml.io .
Name
Stars
Release
Contributors
About
ColossalAI
Making large AI models cheaper, faster and more accessible
Name
Stars
Release
Contributors
About
AutoGPT
An experimental open-source attempt to make GPT-4 fully autonomous.
MetaGPT
๐ The Multi-Agent Framework: Given one line Requirement, return PRD, Design, Tasks, Repo
XAgent
An Autonomous LLM Agent for Complex Task Solving
Name
Stars
Release
Contributors
About
DeepSpeed-MII
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
OpenLLM
Operating LLMs in production
RayServe
Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Triton Inference Server
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
Text Generation Inference
Large Language Model Text Generation Inference
vLLM
A high-throughput and memory-efficient inference and serving engine for LLMs
Name
Stars
Release
Contributors
About
AgentBench
A Comprehensive Benchmark to Evaluate LLMs as Agents
Name
Stars
Release
Contributors
About
deeplake
Database for AI. Store Vectors, Images, Texts, Videos, etc. Use with LLMs/LangChain. Store, query, version, & visualize any AI data. Stream data in real-time to PyTorch/TensorFlow. https://activeloop.ai
Name
Stars
Release
Contributors
About
OpenLLMetry
Open-source observability for your LLM application, based on OpenTelemetry
phoenix
ML Observability in a Notebook - Uncover Insights, Surface Problems, Monitor, and Fine Tune your Generative LLM, CV and Tabular Models