To run the training script, you need to have Python 3.12 and the required packages installed.
pip install -r requirements.txt
Additionally, you need to fill .env file with your Neptune.ai NEPTUNE_PROJECT
and NEPTUNE_API_TOKEN
to log the experiments.
transformers
for obtaining the checkpoints, training loop and evaluationdatasets
for loading the Super GLUE datasetsfast-aug
- our custom library for random data augmentation - written on rust with python bindingsneptune
for logging the experiments (runs available)
To get all the available options, run:
python main.py --help
For example, to train the roberta-base model on the CB task with words substitution augmentation, run:
python main.py --task_name super_glue/cb --model_name roberta-base --augmentation words-sub