Add custom tokens to DNATokenizer
WENHUAN22 opened this issue · 0 comments
WENHUAN22 commented
Hi,
I am wondering is it possible to add tokens to DNATokenizer?
As the function
tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
tokenizer.train_new_from_iterator(owndata)
which can add own tokens to bertTokenizer.