Pinned Repositories
ComfyUI-flash-attention-triton
A ComfyUI node that allows you to select Flash Attention Triton implementation as sampling attention.
exllama
A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
exllamav2
A fast inference library for running LLMs locally on modern consumer-class GPUs
llama.cpp
Port of Facebook's LLaMA model in C/C++
text-generation-webui
A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA.
whisper.cpp
Port of OpenAI's Whisper model in C/C++
llama.cpp
LLM inference in C/C++
whisper.cpp
Port of OpenAI's Whisper model in C/C++
text-generation-webui
A Gradio web UI for Large Language Models.
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
ardfork's Repositories
ardfork/ComfyUI-flash-attention-triton
A ComfyUI node that allows you to select Flash Attention Triton implementation as sampling attention.
ardfork/exllama
A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
ardfork/exllamav2
A fast inference library for running LLMs locally on modern consumer-class GPUs
ardfork/llama.cpp
Port of Facebook's LLaMA model in C/C++
ardfork/text-generation-webui
A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA.
ardfork/whisper.cpp
Port of OpenAI's Whisper model in C/C++