nlp-uoregon/trankit

switch base model from xlm-roberta-base

Closed this issue · 2 comments

Sneriko commented

Hi, I'm working with historical swedish text, and want to train a sentence-segmenter for that (historical swedish text is very inconsistent with punctuation, capital letters etc). I've finetuned a swedish BERT-model on a historical text corpora, and now I want to use this model instead of xlm-roberta-base, when I train the sentence segmenter, I've tried changing the base model, so that it loads the BERT model instead, but I get a mismatch in parameters. Can you give me some tips on what I have to do to change the base_model from xlm-roberta-base to my finetuned historical BERT-model.

Best regards, and thanks for a great repo!

Bachstelze commented

It seems like the xlm-roberta-model is hardcoded into the codebase.
Have a look at spacy for individual models: https://explosion.ai/blog/ud-benchmarks-v3-2

minhhdvn commented

Hi @Sneriko,
Thanks for letting us know.
To support another base model, one way is to clone our repo and install it from source. Once the installation is done, you may change the encoder specified in the two following files: