Build better LLM apps — faster, smarter, production-ready.
A curated, list of 100+ libraries and frameworks for AI engineers building with Large Language Models. This toolkit includes battle-tested tools, frameworks, templates, and reference implementations for developing, deploying, and optimizing LLM-powered systems.
🛠️ Tooling for AI Engineers
Tool
Description
Language
License
Pinecone Managed vector database for production AI applications
API/SDK
Commercial
Weaviate Open-source vector database with GraphQL API
Go
BSD-3
Qdrant Vector similarity search engine with extended filtering
Rust
Apache-2.0
Chroma Open-source embedding database for LLM apps
Python
Apache-2.0
Milvus Cloud-native vector database for scalable similarity search
Go/C++
Apache-2.0
FAISS Library for efficient similarity search and clustering
C++/Python
MIT
Orchestration & Workflows
Tool
Description
Language
License
LangChain Framework for developing LLM applications
Python/JS
MIT
LlamaIndex Data framework for LLM applications
Python
MIT
Haystack End-to-end NLP framework for production
Python
Apache-2.0
DSPy Framework for algorithmically optimizing LM prompts
Python
MIT
Semantic Kernel SDK for integrating AI into conventional programming languages
C#/Python/Java
MIT
Langflow Visual no-code platform for building and deploying LLM workflows
Python/TypeScript
MIT
Flowise Drag-and-drop UI for creating LLM chains and agents
TypeScript
MIT
PDF Extraction Tools
Tool
Description
Language
License
Docling AI-powered toolkit converting PDF, DOCX, PPTX, HTML, images into structured JSON/Markdown with layout, OCR, table, and code recognition
Python
MIT
pdfplumber Drill through PDFs at a character level, extract text & tables, and visually debug extraction
Python
MIT
PyMuPDF (fitz) Lightweight, high-performance PDF parser for text/image extraction and manipulation
Python / C
AGPL-3.0
PDF.js Browser-based PDF renderer with text extraction capabilities
JavaScript
Apache-2.0
Camelot Extracts structured tabular data from PDFs into DataFrames and CSVs
Python
MIT
RAG (Retrieval-Augmented Generation)
Tool
Description
Language
License
RAGFlow Open-source RAG engine based on deep document understanding
Python
Apache-2.0
Verba Retrieval Augmented Generation (RAG) chatbot
Python
BSD-3
PrivateGPT Interact with documents using local LLMs
Python
Apache-2.0
AnythingLLM All-in-one AI application for any LLM
JavaScript
MIT
Quivr Your GenAI second brain
Python/TypeScript
Apache-2.0
Jina Cloud-native neural search framework for multimodal RAG
Python
Apache-2.0
txtai All-in-one embeddings database for semantic search and workflows
Python
Apache-2.0
Tool
Description
Language
License
Ragas Evaluation framework for RAG pipelines
Python
Apache-2.0
LangSmith Platform for debugging, testing, and monitoring LLM applications
API/SDK
Commercial
Phoenix ML observability for LLM, vision, language, and tabular models
Python
Apache-2.0
DeepEval LLM evaluation framework for unit testing LLM outputs
Python
Apache-2.0
TruLens Evaluation and tracking for LLM experiments
Python
MIT
Inspect Framework for large language model evaluations
Python
Apache-2.0
UpTrain Open-source tool to evaluate and improve LLM applications
Python
Apache-2.0
Tool
Description
Language
License
Hugging Face Hub Client library for Hugging Face Hub
Python
Apache-2.0
MLflow Platform for ML lifecycle management
Python
Apache-2.0
Weights & Biases Developer tools for ML
Python
MIT
DVC Data version control for ML projects
Python
Apache-2.0
Comet ML Experiment tracking and visualization for ML/LLM workflows
Python
MIT
ClearML End-to-end MLOps platform with LLM support
Python
Apache-2.0
Data Collection & Web Scraping
Tool
Description
Language
License
Firecrawl AI-powered web crawler that extracts and structures content for LLM pipelines
TypeScript
MIT
Scrapy Fast, high-level web crawling & scraping framework
Python
BSD-3
Playwright Web automation & scraping with headless browsers
TypeScript/Python/Java/.NET
Apache-2.0
BeautifulSoup Easy HTML/XML parsing for quick scraping tasks
Python
MIT
Selenium Browser automation framework (supports scraping)
Multiple
Apache-2.0
Apify SDK Web scraping & automation platform SDK
Python/JavaScript
Apache-2.0
Newspaper3k News & article extraction library
Python
MIT
Framework
Description
Language
License
AutoGen Multi-agent conversation framework
Python
CC-BY-4.0
CrewAI Framework for orchestrating role-playing autonomous AI agents
Python
MIT
LangGraph Build resilient language agents as graphs
Python
MIT
AgentOps Python SDK for AI agent monitoring, LLM cost tracking, benchmarking
Python
MIT
Swarm Educational framework for exploring ergonomic, lightweight multi-agent orchestration
Python
MIT
Agency Swarm An open-source agent framework designed to automate your workflows
Python
MIT
Multi-Agent Systems Research into multi-agent systems and applications
Python
MIT
Auto-GPT Autonomous AI agent for task execution using GPT models
Python
MIT
BabyAGI Task-driven autonomous agent inspired by AGI
Python
MIT
SuperAGI Infrastructure for building and managing autonomous agents
Python
MIT
Phidata Build AI agents with memory, tools, and knowledge
Python
MIT
MemGPT Self-improving agents with infinite context via memory management
Python
MIT
📦 LLM Development & Optimization LLM Training and Fine-Tuning
Tool
Description
Language
License
PyTorch Lightning High-level PyTorch interface for LLMs
Python
Apache-2.0
unsloth Fine-tune LLMs faster with less memory
Python
Apache-2.0
Axolotl Post-training pipeline for AI models
Python
Apache-2.0
LLaMA-Factory Easy & efficient LLM fine-tuning
Python
Apache-2.0
PEFT Parameter-Efficient Fine-Tuning library
Python
Apache-2.0
DeepSpeed Distributed training & inference optimization
Python
MIT
TRL Train transformer LMs with reinforcement learning
Python
Apache-2.0
Transformers Pretrained models for text, vision, and audio tasks
Python
Apache-2.0
LLMBox Unified training pipeline & model evaluation
Python
MIT
LitGPT Train and fine-tune LLMs lightning fast
Python
Apache-2.0
Mergoo Merge multiple LLM experts efficiently
Python
Apache-2.0
Ludwig Low-code framework for custom LLMs
Python
Apache-2.0
txtinstruct Framework for training instruction-tuned models
Python
Apache-2.0
xTuring Fast fine-tuning of open-source LLMs
Python
Apache-2.0
RL4LMs RL library to fine-tune LMs to human preferences
Python
Apache-2.0
torchtune PyTorch-native library for fine-tuning LLMs
Python
BSD-3
Accelerate Library to easily train on multiple GPUs/TPUs with mixed precision
Python
Apache-2.0
BitsandBytes 8-bit optimizers and quantization for efficient LLM training
Python
MIT
Open Source LLM Inference
Tool
Description
Language
License
LLM Compressor Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment
Python
Apache-2.0
LightLLM Lightweight Python-based LLM inference and serving framework with easy scalability and high performance
Python
Apache-2.0
vLLM High-throughput and memory-efficient inference and serving engine for LLMs
Python
Apache-2.0
torchchat Run PyTorch LLMs locally on servers, desktop, and mobile
Python
MIT
TensorRT-LLM NVIDIA library for optimizing LLM inference with TensorRT
C++/Python
Apache-2.0
WebLLM High-performance in-browser LLM inference engine
TypeScript/Python
Apache-2.0
Tool
Description
Language
License
JailbreakEval Automated evaluators for assessing jailbreak attempts
Python
MIT
EasyJailbreak Easy-to-use Python framework to generate adversarial jailbreak prompts
Python
Apache-2.0
Guardrails Add guardrails to large language models
Python
MIT
LLM Guard Security toolkit for LLM interactions
Python
Apache-2.0
AuditNLG Reduce risks in generative AI systems for language
Python
MIT
NeMo Guardrails Toolkit for adding programmable guardrails to LLM conversational systems
Python
Apache-2.0
Garak LLM vulnerability scanner
Python
MIT
DeepTeam LLM red teaming framework
Python
Apache-2.0
AI App Development Frameworks
Tool
Description
Language
License
Reflex Build full-stack web apps powered by LLMs with Python-only workflows and reactive UIs.
Python
Apache-2.0
Gradio Create quick, interactive UIs for LLM demos and prototypes.
Python
Apache-2.0
Streamlit Build and share AI/ML apps fast with Python scripts and interactive widgets.
Python
Apache-2.0
Taipy End-to-end Python framework for building production-ready AI apps with dashboards and pipelines.
Python
Apache-2.0
Local Development & Serving
Tool
Description
Language
License
Ollama Get up and running with large language models locally
Go
MIT
LM Studio Desktop app for running local LLMs
-
Commercial
GPT4All Open-source chatbot ecosystem
C++
MIT
LocalAI Self-hosted OpenAI-compatible API
Go
MIT
Platform
Description
Pricing
Features
Clarifai Lightning-fast compute for AI models & agents
Free tier + Pay-as-you-go
Pre-trained models, Deploy your own models on Dedicated compute, Model training, Workflow automation
Modal Serverless platform for AI/ML workloads
Pay-per-use
Serverless GPU, Auto-scaling
Replicate Run open-source models with a cloud API
Pay-per-use
Pre-built models, Custom training
Together AI Cloud platform for open-source models
Various
Open models, Fine-tuning
Anyscale Ray-based platform for AI applications
Enterprise
Distributed training, Serving
We welcome contributions! This toolkit grows stronger with community input.
Fork the repository Create a feature branch (git checkout -b feature/amazing-tool)Add your contribution (new tool, template, or tutorial)Submit a pull request
Quality over quantity - Focus on tools and resources that provide real valueProduction-ready - Include tools that work in real-world scenariosWell-documented - Provide clear descriptions and usage examplesUp-to-date - Ensure tools are actively maintained
Get weekly AI engineering insights, tool reviews, and exclusive demos and AI Projects delivered to your inbox:
📧 Subscribe to AI Engineering Newsletter →
Join 100,000+ engineers building better LLM applications
Built with ❤️ for the AI Engineering community
Star ⭐ this repo if you find it helpful!
