KeyError with HuggingFace NER Pipeline

[tbabinet]

Hello,
I've been trying to use scibert_scivocab_uncased with the NER pipeline from HuggingFace, but I can't seem to make it work. Each time I try to run the ner pipeline, my script crashes. The code is very simple :
nlp(abstract)
and the error is the following :

KeyError Traceback (most recent call last)
in
3 if(len(row.abstract.split())<512):
4 abst = row.abstract
----> 5 print(nlp(abst))
6
7 except ValueError as ve:

c:\users\tbabi\appdata\local\programs\python\python37\lib\site-packages\transformers\pipelines.py in call(self, *texts, **kwargs)
792 answer = []
793 for idx, label_idx in enumerate(labels_idx):
--> 794 if self.model.config.id2label[label_idx] not in self.ignore_labels:
795 answer += [
796 {

KeyError: 422

I have been struggling with this for quiet a time now. Has anyone else been bumping into this and could give some help ?

Thank you !

[stefan-it]

@tbabinet Could you provide a full code snippet?

If you want to use the NER pipeline, then you need to use a fine-tuned model of e.g. SciBERT (the "normal" SciBERT model from model hub can't be used)

[tbabinet]

Hello
It seems I had simply been misusing my model : I was instantiating an AutoModel, when I should have been using the BertForTokenClassification class.
Thanks anyway ! :)

[ibeltagy]

glad you figured it out.