wenqiglantz/llmops

A collection of notebooks related to LLMOps

LLMOps

Inspired by Maxime Labonne's llm-course, this repo holds a collection of my experiments of LLM quantization, merge, evaluation, and fine-tuning.

The Colab notebooks are referenced in my blog posts:

Democratizing LLMs: 4-bit Quantization for Optimal LLM Inference

• Quantize_Mistral_7B_Instruct_v0_2_using_GGUF_and_llama_cpp.ipynb
• Eval_for_quantized_models_for_Mistral_7B_Instruct_v0_2.ipynb
• Eval_for_base_model_of_Mistral_7B_Instruct_v0_2.ipynb

Exploring mergekit for Model Merge, AutoEval for Model Evaluation, and DPO for Model Fine-tuning

• LazyMergekit.ipynb
• LLM_AutoEval_MistralTrinity_7b_slerp.ipynb
• FineTuning_Dataset_DataBricks_Mistral7B.ipynb
• Fine_tune_MistralTrinity_7b_slerp_with_QLoRA_and_Databricks_dataset.ipynb
• Fine_tune_MistralTrinity_7B_slerp_with_DPO.ipynb
• Merge_Family_Tree_MistralTrinity_7B_slerp_dpo.ipynb