Pinned Repositories
AutoAWQ
AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference.
vllm-gptq
A high-throughput and memory-efficient inference and serving engine for LLMs
ctransformers
Python bindings for the Transformer models implemented in C/C++ using GGML library.
llm-awq
[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
lightllm
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
curname's Repositories
curname/AutoAWQ
AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference.
curname/vllm-gptq
A high-throughput and memory-efficient inference and serving engine for LLMs