makcedward/nlpaug

The tokenizer class you load from this checkpoint is 'RobertaTokenizer'

bhomass opened this issue · 0 comments

bhomass commented

I ran thru the lambada-train_model.ipynb example on my data, and got the tokenizer error

The tokenizer class you load from this checkpoint is 'RobertaTokenizer'. 
The class this function is called from is 'GPT2Tokenizer'.

There is no mention of tokenizer in any of the call parameters. How did this tokenizer mix up occur?