/BiGS

Official Repository of Pretraining Without Attention (BiGS), BiGS is the first model to achieve BERT-level transfer learning on the GLUE benchmark with subquadratic complexity in length (or without attention).

Primary LanguagePythonApache License 2.0Apache-2.0

Stargazers