exllama

There are 12 repositories under exllama topic.

gotzmann/booster
Booster - open accelerator for LLM models. Better inference and debugging for AI hackers
Language:C++145 7 56
c0sogi/llama-api
An OpenAI-like LLaMA inference API
Language:Python113 4 229
innightwolfsleep/text-generation-webui-telegram_bot
LLM telegram bot
Language:Python108 6 6221
shinomakoi/magi_llm_gui
A Qt GUI for large language models
Language:Python40 3 134
shinomakoi/AI-Messenger
A QT GUI for large language models
Language:Python24 3 05
silphendio/sliced_llama
Simple LLM inference server
Language:Python19 3 00
NO-ob/simpleLlama
A Simple webserver for generating text with exllamav2
Language:Python14 2 02
Aqirito/A.L.I.C.E
A.L.I.C.E (Artificial Labile Intelligence Cybernated Existence). A REST API of A.I companion for creating more complex system
Language:Python7 3 00
seyf1elislam/LocalLLM_OneClick_Colab
Run gguf LLM models in Latest Version TextGen-webui
Language:Jupyter Notebook7 1 00
kooten111/EasyEXL
A Python script designed to streamline the process of quantizing models to exllamav2 format
Language:Python0 2 02
alexkreidler/quote-constraint
A constrained generation filter for local LLMs that makes them quote properly from a source document
Language:Python1 0
countzero/windows_exllama
This is a playground to explore the ExLlama project in a Windows environment.
Language:PowerShell2 0