vdutts7/mistral-7b-finetune

Mistral 7B Finetune

Fine-tuning Mistral 7B model using the UltraChat dataset

Table of Contents

📝 About
📊 Dataset
💻 How to Run
🔧 Tools Used
👤 Contact

📝About

This project demonstrates the process of fine-tuning the Mistral 7B language model using the UltraChat dataset. It covers the entire pipeline from downloading the model to inference with the fine-tuned model.

📊Dataset

The project uses the UltraChat dataset from Hugging Face:

• Dataset: HuggingFaceH4/ultrachat_200k
• Specific file used: test_gen-00000-of-00001-3d4cd8309148a71f.parquet

💻 How to Run

1. Download the Model

   • Clone the Mistral finetune repository
   • Install requirements
   • Download the Mistral 7B model

2. Prepare the Dataset

   • Download and split the UltraChat dataset
   • Reformat the data for training

3. Configure Training

   • Set up the training configuration in a YAML file

4. Start Training

   • Run the training script

5. Inference

   • Load the fine-tuned model and run inference

For detailed steps and code, please refer to the Jupyter notebook in this repository.

🔧Tools Used

👤Contact