A potential bug that may mask the start and stop tokens
Opened this issue · 1 comments
lishen commented
Line 351 in 3d3657b
If I understand your code correctly, this line shall be continue instead of pass. If you use pass, the code will continue to mask the start and stop tokens with slightly less than 15% chance. I'm a bit surprised this hasn't caused any serious issues.
rmrao commented
Oh oops. Shouldn't really matter - predicting the start and stop tokens is easy enough for the model. I'll update the code when I get the chance.