A question about inference
itaim opened this issue · 6 comments
Hi,
I have a question about inference.
I tried running inference at the end of the notebook
import torch
model_inputs = test_dataset[0]
print(model_inputs.keys())
print(model_inputs['numerical_feats'].size())
with torch.no_grad():
_, logits, classifier_outputs = model(
model_inputs['input_ids'],
token_type_ids=model_inputs['token_type_ids'],
cat_feats=model_inputs['cat_feats'],
numerical_feats=model_inputs['numerical_feats']
)
But I get an error:
ValueError: Wrong shape for input_ids (shape torch.Size([156])) or attention_mask (shape torch.Size([156]))
for numerical_feats which size is torch.Size([3])
. What am I doing wrong? Could you provide a more detailed example of how to run inference and what and if there is additional preprocessing to be done?
I also get error for inference. My error is also for numerical feature. I tried both CPU and GPU.
with torch.no_grad():
_, logits, classifier_outputs = model(
torch.tensor([test_dataset.encodings['input_ids'][0]]).to(device),
torch.tensor([test_dataset.encodings['token_type_ids'][0]]).to(device),
cat_feats=torch.tensor([test_dataset.cat_feats[0]]).to(device),
numerical_feats=torch.tensor([test_dataset.numerical_feats[0]]).to(device)
)
Here is the error I get:
2057 input, weight, bias, running_mean, running_var,
-> 2058 training, momentum, eps, torch.backends.cudnn.enabled
2059 )
2060
RuntimeError: expected scalar type Double but found Float```
My numerical feature is of type float64. What is the compatible data type?
I also have problem in inferencing. An example would be appreciated.
i have the same issue while inference.
i have the same issue while inference.
Me too, Anybody solve this issue?
@itaim Maybe you should add an extra dimension for each input, then it works
import torch
model_inputs = test_dataset[0]
with torch.no_grad():
_, logits, classifier_outputs = model(
model_inputs['input_ids'].unsqueeze(0).cuda(),
token_type_ids=model_inputs['token_type_ids'].unsqueeze(0).cuda(),
cat_feats=model_inputs['cat_feats'].unsqueeze(0).cuda(),
numerical_feats=model_inputs['numerical_feats'].unsqueeze(0).cuda()
)
@cultivater has it right! You need to add an extra dimension to each input since the model expects a batch of data. In this case, we send in a batch of 1.