Why dose the results significantly different between CPT and DAS.
Closed this issue · 3 comments
As the title mentioned, the result from your two papers is significantly different, just like RoBERTa on Restaurant dataset, while DAS get 79.81 scores on MF1, and the CPT just only 50.61. Could you tell me why or if I make misunderstood the task setting? Thanks!
CPT:
DAS:
Hello, Allen! Thanks for your question.
In CPT paper, we conduct experiments using few-shot learning end-tasks. While in DAS paper, we use the full dataset. That's why the results in CPT paper are significantly lower than those in DAS paper. You can check our CPT paper for the details of the few-shot learning setting.
Hope the comments above will address your problem.
Let me close this issue. If you have other questions, you can open another issue :)
OK, thanks! I have read the paper of CPT again and find the few-shot setting in the footnote as following picture show.
