Match indices out of range
llschloesser opened this issue · 2 comments
I appreciate your work, it appears to be a nice improvement over SuperGlue. I am using superpoint_lightglue.trt.onnx.
However, I occasionally see match indices significantly out of range (1024 keypoints are fed in) and I am hoping you can shed some light on why this might be happening. For example I will see output like the following,:
19:35:49.135455 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1049042944
19:35:49.135459 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1050992640
19:35:49.135460 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1061773312
19:35:49.135460 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1046601728
19:35:49.135462 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1040564224
19:35:49.135463 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1044709376
19:35:49.135464 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1043808256
19:35:49.135465 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1044045824
19:35:49.135466 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1048846336
19:35:49.135466 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1040687104
19:35:49.135467 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1048068096
19:35:49.135468 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1054064640
19:35:49.135468 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1044054016
19:35:49.135470 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1040228352
19:35:49.135470 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1052934144
19:35:49.135471 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1049477120
19:35:49.135472 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1041784832
19:35:49.135473 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1040883712
19:35:49.135474 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1058586624
19:35:49.135474 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1059799040
19:35:49.135475 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1038467072
19:35:49.135476 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1038794752
19:35:49.135477 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1040367616
19:35:49.135478 WARNING[TensorRTLightGlue.cpp>match:105] Query index out of range: 1042014208
I detect and throw away these matches in the output and performance seems unaffected, but why might this be happening? Is there an iterpretation of these values that I am missing?
...
model_->infer(inputs);
// obtain views of output data
auto [dims0, data0] = model_->outputs()[0];
cv::Mat indices(1, dims0[0], CV_32SC2, data0); // N_matches x 2
auto [dims1, data1] = model_->outputs()[1];
cv::Mat scores(1, dims1[0], CV_32FC1, data1); // N_matches
matches.clear();
matches.reserve(scores.cols);
auto scores_ptr = scores.ptr<float>();
auto indices_ptr = indices.ptr<cv::Vec2i>();
for (size_t i = 0; i < scores.cols; ++i) {
if (indices_ptr[i][0] < 0 or indices_ptr[i][0] >= query_keypoints.size()) {
LOG(WARNING) << "Query index out of range: " << indices_ptr[i][0];
continue;
}
if (indices_ptr[i][1] < 0 or indices_ptr[i][1] >= train_keypoints.size()) {
LOG(WARNING) << "Train index out of range: " << indices_ptr[i][1];
continue;
}
matches.emplace_back(indices_ptr[i][0], indices_ptr[i][1], scores_ptr[i]);
}
...Thanks!
Hi @llschloesser, thank you for your interest in LightGlue-ONNX.
Could you provide some sample inputs so that I can try to reproduce this? Does the problem still occur when using this model: https://github.com/fabio-sim/LightGlue-ONNX/releases/download/v2.0/superpoint_lightglue_pipeline.trt.onnx ?
This was an output array allocation error on our side. Works great now, thanks again for this repo.