elyase/awesome-gpt3

GPT vs BERT, under same computation and data resource, which one is better for downstream tasks like GLUE?

guotong1988 opened this issue · 1 comments

guotong1988 commented

Thank you very much.

EricLee8 commented

感觉BERT之类的用auto encoder模式来预训练的模型更适合GLUE吧，GPT比较擅长text generation