bloomsburyai/question-generation

Which tokenizer does this model use?

Closed this issue · 1 comments

zzj0402 commented

As title.

tomhosking commented

Tokenisation happens here - it uses the NLTK tokenizer.