WaterPistolAI/Awesome-Local-LLM

A curated list of resources, libraries, tools, and communities for working with Local Large Language Models (LLMs).

Awesome List for Local LLMs

A curated list of resources, libraries, tools, and communities for working with Local Large Language Models (LLMs).
Join us on Discord!

Model Libraries

• Hugging Face Website | Discord | Docs
• ModelScope Website | Discord | Docs
• Ollama Website | Git | Discord | Docs
• Github Models Website | Docs

UI

• Open WebUI Website | Git | Discord | Docs
• Anything LLM Website | Git | Discord | Docs
• ComfyUI Website | Git | Matrix | Docs
• oterm Git

Local Inference

• Llama.cpp Git | Docs
• Ollama Website | Git | Discord | Docs
• Distributed Llama Git | Discord | Docs
• vLLM Git | Discord | Docs
• LM Studio Website | Discord | Docs

Coding

• Twinny Website | Git | Docs
• Continue Website | Git | Docs
• Open Hands Git | Discord | Docs
• SWE-Agent Website | Git | Discord | Docs
• Aider Git

Terminal

• Self-Operating Computer Website | Git | Discord | X
• Neo-AI Git Docs
• Warp Website | Git | Discord | Docs

Agents

• AG2 Website Git | Discord | Docs
• Autogen Website | Discord | Docs
• Crew AI Website | Git | Docs
• Ollama Agents Git
• Agent Chef Discord | Git | Docs
• Rollcage Discord | Git | Docs
• Agentless Git

Diffusion

• PyTorch Website | Git | Discord | Docs
• Stable Diffusion Website | Git | Discord
• CivitAI Website | Discord | Docs

Search and Scrape

• SearXNG Website | Git | Matrix
• Apache Tika Website | Git
• Meilisearch Website | Git | Discord | Docs
• FireCrawl Website | Git | Docs
• RagArena Website | Git | Docs
• RSSHub Website | Git | Docs
• Raycast Website | Git | Docs
• Fresh RSS Website | Git | Docs
• Browserless Website | Git | Docs
• Scrapy Website | Git | Docs

TTS

• Whisper Git
• Whisper.cpp Git
• Faster Whisper Git
• Faster Whisper Server https://github.com/fedirz/faster-whisper-server

STT

• OpenedAI Speech Git | Docs
• Coqui Website | Git
• LLaMa Omni Git

Datasets

• Datasets Git | Docs
• Dataset Viewer Git | Docs
• Papers with Code Datasets
• Ollama Grid Search Git

Training

• Autotrain-Advanced Website | Git | Docs
• LLaMA Factory Website | Git | Docs
• Unsloth Website | Git | HF | Discord | Docs
• Liger Kernel Git | Discord
• MergeKit Website | Git | Docs
• Axolotl Website | Git | Discord | Docs
• Abliterator Git

Chat and API

• ChatGPT Website | Docs
• Cursor Website | Git | X
• Claude Website | Docs
• Mistral AI Website | Discord | Docs
• Mirage Website | Git | Discord | Docs
• Midjourney Website
• HyperWrite Website

VPS and Instance

• Hugging Face Spaces Website | Git | Discord | Docs
• LightningAI Website | Discord | Docs
• Intel DevCloud Website | Docs
• NVIDIA NIM Website | Docs
• Amazon SageMaker Website | Docs
• Google Colab Website
• Groq Website

General

• Llama.cpp Git | Docs
• CUDA Toolkit Website | Docs
• ROCm Git | Docs
• AMD SHARK Discord
• Python Discord
• Liquid Neural Network Git
• Genesis Embodied AI Git

Community

• BORCH's UToE Git | Discord
• The Drummer/BeaverAI HF | Discord

Reference

• Ollama and Agents
• GraphRAG + AutoGen + Ollama + Chainlit UI
• Awesome Public Datasets
• Awesome Alternative Frontends
• Awesome Torch
• Train a Llama Model from Scratch
• SAELens Example Dashboard Link

Resources

• DataCamp https://www.datacamp.com

  • Speculative Decoding Tutorial

• ComfyUI

  • https://comfyui-wiki.com/
  • https://comfyui-wiki.com/news/2024-10-07-jasperai-releases-flux-dev-controlnet-models

• FLUX

  • https://comfyui-wiki.com/tutorial/advanced/flux1-comfyui-guide-workflow-and-examples.en-US
  • https://github.com/city96/ComfyUI-GGUF
  • https://huggingface.co/city96/FLUX.1-dev-gguf
  • https://huggingface.co/city96/FLUX.1-schnell-gguf
  • https://huggingface.co/city96/t5-v1_1-xxl-encoder-gguf

Contribution

Contributions are welcome! If you have a resource that would fit into this list, please submit a pull request or open an issue describing the resource and why it should be included.

License

This list is provided under the CC0 1.0 Universal (CC0 1.0) Public Domain Dedication. Feel free to use, share, modify, and distribute this list as you see fit.