JetBrαins interηship 2023 test assignment repository. The "Aηalysis of interηal representαtions in code generαtioη models" problem. (the special symbols added to remove repo from GitHub search)
train.py
- script to finetune any HuggingFace-based model usingparams.json
parameters;params.json
- set of parameters for finetuning the DL models. The param set fall back to its prefix (e.g.default-b8
will extenddefault
);report.ipynb
- report notebook, allows to run the training script in colab and provide some error analysis;
For the training parameters see python train.py --help
option. Example:
python train.py --config-name=default --base-model=t5-small --push-to-pub
Required: Python version 3.9 as colab have this version at the moment.
For the local usage venv usage is recommended:
python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
- The project uses neptune.ai for experiments tracking. So, the training scripts require
NEPTUNE_PROJECT
andNEPTUNE_API_TOKEN
environment variables to be set.
-
For Local usage:
Each.py
file provided is standalone training script required onlyparams.json
file to operate.
It can be run withpython train.py
command. For the parameters see--help
option. -
For Colab usage: see
report.ipynb
notebook or online colab copy.
This notebook only provides report and place to runtrain.py
file to use colab gpu.
The requiredtrain.py
files andparams.json
have to be copied in thecontent/
folder.
The full report is available in report.ipynb
or online colab copy.
All experiments metrics for training are available in neptune.ai project.
tl;dr: The best model achieved BLEU: 34.96, EM: 21.4
(better than authors model, worse than top-1) on CodeXGLUE ConCode
benchmark. Salesforce/codet5-small
model was trained with w_special_tokens
config from params.json
using train.py
script.
The final model is available at HuggingFace Hub as k4black/Salesforce-codet5-small-CodeXGLUE-CONCODE-w_special_tokens`.