An easy-to-use LLM quantization and inference toolkit based on GPTQ algorithm (weight-only quantization).
Primary LanguagePythonApache License 2.0Apache-2.0
No one’s star this repository yet.