lucidrains/electra-pytorch
A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch
PythonMIT
Issues
- 0
pip install electra-pytorch or python setup.py test and wrong occured like this
#15 opened by mengyougithub - 1
Disallow Sampling From Correct
#12 opened by zanussbaum - 0
- 0
Custom Dataset
#9 opened by appledora - 0
- 1
Electra-small performance
#6 opened by YovaKem - 0
nan loss during pretraining
#7 opened by bakuzen - 1
- 1
Simplest way to modify owt pretraining to use ReformerLM as generator and discriminator
#4 opened by DarrenAbramson - 5
- 1