jerryji1993/DNABERT

Add custom tokens to DNATokenizer

WENHUAN22 opened this issue · 0 comments

Hi,
I am wondering is it possible to add tokens to DNATokenizer?

As the function
tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
tokenizer.train_new_from_iterator(owndata)
which can add own tokens to bertTokenizer.