/quantize-hf-models

Quantize LLMs from HuggingFace into the GGUF format (both standard and with imatrix) using a Colab notebook.

Primary LanguageJupyter NotebookApache License 2.0Apache-2.0

Quantize Hf Models

Quantize LLMs from HuggingFace into the GGUF format (both standard and with imatrix) using a Colab notebook.

License

Apache 2.0 License