VincentK1991/BERT_summarization_1

RunTime error at training step

abdullah-abunada opened this issue · 5 comments

abdullah-abunada commented

Hello,

I have this issue in training step
RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.FloatTensor instead (while checking arguments for embedding)

this issue raised at File "train_GPT2.py", line 105.
outputs = model(input_ids = input_ids, mc_token_ids = mc_token_ids, mc_labels = mc_labels, lm_labels = lm_labels, token_type_ids = token_type_ids)

can you check it, please!

abdullah-abunada commented

I solve it by converting input_ids to long

input_ids = input_ids.to(torch.long)

But this issue raised IndexError: index out of range in self
torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)

  • 👍1
eva806 commented

I'm also facing same issue after applying conversion.
IndexError: index out of range in self
torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse

Did you get any solution?

manastahir commented

Adding the line corrects the issue.
tokenizer.add_special_tokens(special_tokens)

Akash860249 commented

Adding the line corrects the issue.
tokenizer.add_special_tokens(special_tokens)

Where to add that line?

Akash860249 commented

@VincentK1991 Can you please share model,config and other files that are obtained after training (train_GPT2.py).
Thanks in Advance.