The input of RoBERTa model after labelling some articles for the step ordering task
WangRuoke opened this issue · 1 comments
We ask 3 annotators to label 1,000 wikiHow articles as ordered or not as a coarse-grained approximation for whether their steps are ordered. We finetune a pre-trained RoBERTa model using 5- fold cross-validation, finding an average precision of 88%. We then ask a 4th annotator to label another 40 articles as the held-out test set, where the finetuned model achieves 100% precision. Finally, we only consider articles that the model predicts as ordered (around 40%) for the Step Ordering task.
In this part, you have mentioned using a model to predict whether the article is ordered and we need to use a pre-trained RoBERTa model, I would like to ask about the specific input of RoBERTa model.
Thx!
Corresponded via email. Marked as closed.