Jwoo5/fairseq-signals

Pretrained model selection

Closed this issue · 2 comments

sehunfromdaegu commented

Hi,

I couldn't find information on how many epochs are needed for the pretraining model in your paper. Is it same as in the config in this repository?

Additionally, are downstream tasks performed using the last checkpoint of the pretrained model, or is the best epoch checkpoint used?

Thank you.

Jwoo5 commented
  1. yes, the number of epochs that we used for pretraining is the same with the specified epochs in each config file.
  2. we used the best epoch checkpoint where the relevant metric for the validation set is the best. you can see that the configurations for this option are commented out in the config files.