XLNet generates random word embeddings
Opened this issue · 4 comments
Hi,
Setting Dropout in production results in random embeddings. This has been discussed in the main repo if you search. To reproduce this, just run the same string more than once, each time you'll see different embeddings.
@maziyarpanahi is this solved??
If the linked PR bing merged which sets Dropout to 0.0 then the random embeddings will be resolved as the authors of the XLNet suggested.
@maziyarpanahi so if I wanted to get XLNet embeddings now what should I do? Wait till they merge or is there any other way
Hey sorry, we kind of got busy with other things and totally forgot about this. I'll test out my PR today and get it merged. In the meanwhile if you can test out the patch in the PR and confirm that will also help