/fine-tuning-sentence-transformers

A Very Simple Demo of Fine Tuning Sentence Transformers

Primary LanguagePythonMIT LicenseMIT

fine-tuning-sentence-transformers

A Very Simple Demo of Fine Tuning Sentence Transformers

This repository provides a practical demonstration of how to fine-tune a Sentence Transformer model on a custom dataset and then use the fine-tuned model to generate sentence embeddings. The scripts utilize the PyTorch library and Sentence Transformers for this purpose.

This project is a simple example of how to fine-tune a Sentence Transformer model. It is not designed for large-scale or real-world applications.

Setup and Installation

To set up and run the example, follow these steps:

  1. Clone this repository: git clone https://github.com/adiekaye/fine-tuning-sentence-transformers.git
  2. Navigate to the project directory: cd sentence-transformer-tuning
  3. Create a virtual environment: python -m venv venv
  4. Activate the virtual environment:
  • For Windows: venv\Scripts\activate
  • For macOS/Linux: source venv/bin/activate
  1. Install the required libraries: pip install -r requirements.txt
  2. Run the script 01_tuning_your_model.py to fine-tune the model: python 01_tuning_your_model.py
  3. Run the script 02_using_your_model.py to use the fine-tuned model: python 02_using_your_model.py
  4. To exit the virtual environment, type deactivate in the terminal.

Note: The virtual environment and requirements installation steps are optional but recommended to ensure compatibility and avoid conflicts with other Python packages you may have installed.

Contents

  • 01_tuning_your_model.py: Script that fine-tunes a Sentence Transformer model on a custom dataset.
  • 02_using_your_model.py: Script that uses the fine-tuned Sentence Transformer model to generate sentence embeddings and calculate their cosine similarity.
  • requirements.txt: Lists the required libraries for this project.
  • README.md: Provides instructions for setting up and running the example, and explains the contents of the repository.
  • .gitignore: A simple Git configuration file to ignore the virtual environment directory and other non-essential files.
  • /tuned_models: A directory to store your fine tuned models.
  • /tuned_models/.gitignore: A gitignore file to make sure you don't accidentally commit your fine tuned model.

Usage

The scripts 01_tuning_your_model.py and 02_using_your_model.py are executable as they are. However, you might need to adjust the path of the fine-tuned model in 02_using_your_model.py depending on your directory structure.

The 01_tuning_your_model.py script will train a Sentence Transformer model using the specified training examples and then save the fine-tuned model.

The 02_using_your_model.py script will load the original and the fine-tuned models to generate embeddings for specific sentences, and it will print the cosine similarity between the original and fine-tuned embeddings for each sentence.

License

This project is licensed under the MIT License.