Quantize LLMs from HuggingFace into the GGUF format (both standard and with imatrix) using a Colab notebook.
Apache 2.0 License
Quantize LLMs from HuggingFace into the GGUF format (both standard and with imatrix) using a Colab notebook.
Jupyter NotebookApache-2.0