llmops

There are 441 repositories under llmops topic.

  • dify

    langgenius/dify

    Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.

    Language:TypeScript87.5k5838.6k13k

  • vllm-project/vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Language:Python43k3557.5k6.5k

  • ComposioHQ/composio

    Composio equip's your AI agents & LLMs with 100+ high-quality integrations via function calling

    Language:Python24.7k502564.4k
  • llm-app

    pathwaycom/llm-app

    Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

    Language:Jupyter Notebook22.9k4617393
  • serve

    jina-ai/serve

    ☁️ Build multimodal AI applications with cloud-native stack

    Language:Python21.5k2151.9k2.2k

  • BerriAI/litellm

    Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

    Language:Python19.8k1084.8k2.5k
  • SuperAGI

    TransformerOptimus/SuperAGI

    <⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

    Language:Python16.1k1724171.9k

  • liguodongiot/llm-action

    本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

    Language:HTML15.8k135271.8k

  • raga-ai-hub/RagaAI-Catalyst

    Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

    Language:Python15.7k26383.8k

  • bentoml/OpenLLM

    Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

    Language:Python11.1k57272702
  • langfuse

    langfuse/langfuse

    🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

    Language:TypeScript9.9k311.1k908
  • metaflow

    Netflix/metaflow

    Build, Manage and Deploy AI/ML Systems

    Language:Python8.7k294695822

  • explodinggradients/ragas

    Supercharge Your LLM Application Evaluations 🚀

    Language:Python8.6k421k876

  • dataelement/bisheng

    BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.

    Language:Python7.9k6052001.3k
  • BentoML

    bentoml/BentoML

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    Language:Python7.5k741.1k829
  • gateway

    Portkey-AI/gateway

    A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

    Language:TypeScript7.5k39449555

  • promptfoo/promptfoo

    Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.

    Language:TypeScript6k20834490

  • comet-ml/opik

    Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

    Language:Python5.9k56219413

  • evidentlyai/evidently

    Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

    Language:Jupyter Notebook5.9k48430655

  • traceloop/openllmetry

    Open-source observability for your LLM application, based on OpenTelemetry

    Language:Python5.6k8223713
  • phoenix

    Arize-ai/phoenix

    AI Observability & Evaluation

    Language:Jupyter Notebook5.2k373.3k383
  • superduper

    superduper-io/superduper

    Superduper: End-to-end framework for building custom AI applications and agents.

    Language:Python5k441.3k493

  • tensorchord/Awesome-LLMOps

    An awesome & curated list of best LLMOps tools for developers

    Language:Shell4.6k769456
  • zenml

    zenml-io/zenml

    ZenML 🙏: The bridge between ML and Ops. https://zenml.io.

    Language:Python4.5k43314496
  • giskard

    Giskard-AI/giskard

    🐢 Open-Source Evaluation & Testing for AI & LLM systems

    Language:Python4.4k33471305
  • cube-studio

    tencentmusic/cube-studio

    cube studio开源云原生一站式机器学习/深度学习/大模型AI平台，支持sso登录，大数据平台对接，notebook在线开发，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务VGPU，边缘计算，标注平台，自动化标注，大模型微调，vllm大模型推理，llmops，私有知识库，AI模型应用商店，支持模型一键开发/推理/微调，支持国产cpu/gpu/npu芯片，支持RDMA，支持pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/spark/ray/volcano分布式，deepseek训练推理

    Language:Jupyter Notebook4.1k72154715
  • cognita

    truefoundry/cognita

    RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry

    Language:Python4k3449328

  • decodingml/llm-twin-course

    🤖 𝗟𝗲𝗮𝗿𝗻 for 𝗳𝗿𝗲𝗲 how to 𝗯𝘂𝗶𝗹𝗱 an end-to-end 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻-𝗿𝗲𝗮𝗱𝘆 𝗟𝗟𝗠 & 𝗥𝗔𝗚 𝘀𝘆𝘀𝘁𝗲𝗺 using 𝗟𝗟𝗠𝗢𝗽𝘀 best practices: ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 12 𝘩𝘢𝘯𝘥𝘴-𝘰𝘯 𝘭𝘦𝘴𝘴𝘰𝘯𝘴

    Language:Python3.8k7321622

  • Helicone/helicone

    🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

    Language:TypeScript3.5k20182349

  • 0xPlaygrounds/rig

    ⚙️🦀 Build portable, modular & lightweight Fullstack Agents

    Language:Rust3.3k36114339

  • iusztinpaul/hands-on-llms

    🦖 𝗟𝗲𝗮𝗿𝗻 about 𝗟𝗟𝗠𝘀, 𝗟𝗟𝗠𝗢𝗽𝘀, and 𝘃𝗲𝗰𝘁𝗼𝗿 𝗗𝗕𝘀 for free by designing, training, and deploying a real-time financial advisor LLM system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 𝘷𝘪𝘥𝘦𝘰 & 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴

    Language:Jupyter Notebook3.2k4821524
  • tensorzero

    tensorzero/tensorzero

    TensorZero creates a feedback loop for optimizing LLM applications — turning production data into smarter, faster, and cheaper models.

    Language:Rust3.2k32537204

  • PacktPublishing/LLM-Engineers-Handbook

    The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices

    Language:Python3k4125632

  • Josh-XT/AGiXT

    AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.

    Language:Python3k71414394
  • pezzo

    pezzolabs/pezzo

    🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

    Language:TypeScript2.8k27115241

  • OpenPipe/OpenPipe

    Turn expensive prompts into cheap fine-tuned models

    Language:TypeScript2.6k2057136