/gpt-gmlp-experiments

The new gMLP model was proposed recently, but it was only applied to BERT. The report aims to understand whether gMLPs can perfectly replace the functionality of Tranformers in GPT and evaluate the features of gMLP announced by Liu et al..

Primary LanguagePythonMIT LicenseMIT

Stargazers