IndexError: index out of range in self
Opened this issue · 3 comments
Siomarry commented
请问一下,在某些输入数据中,产生了这样的问题.
Traceback (most recent call last):
File "D:/Bert+Once/CLUENER2020/BERT-CRF/CheckBug.py", line 14, in
embed = embedding(input_to_embed)
File "D:\Bert+Once\venv\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "D:\Bert+Once\venv\lib\site-packages\torch\nn\modules\sparse.py", line 145, in forward
return F.embedding(
File "D:\Bert+Once\venv\lib\site-packages\torch\nn\functional.py", line 1913, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self
是什么原因呢.
zjmwqx commented
空格和tab token不会被识别。导致label和tokens的数量不一致。你看句子里是不是有。需要去掉。作者这里用了一种暴力的对齐方式。我再看有没有更好的办法