Reasons why pre-learning was not effective

Question

Opened this issue 2 years ago · 0 comments

Pre-training the transformer seems like an effective idea. Do you have any theories as to why it didn't work?