fabio-sim/LightGlue-ONNX

convert SuperPoint from onnx to engine

midskymid opened this issue · 1 comments

i tried to convert SperPoint onnx model to tensorrt engine by FP16, and i met trouble:
i used SuperPoint_FP16.engine to extract features and LightGlue_FP16.engine to match them, but the results was bad and could not perform as well as onnx model both of them. did you meet the same problem? and how to solve it?
1717517141647

Hi @midskymid, thank you for your interest in LightGlue-ONNX.

If I recall correctly, I do think I've encountered a similar problem during my own experimentation. Depending on how you convert it to half-precision, it can happen that the values go outside the tolerable floating point range, leading to a drastic fall in # of keypoints detected, or even none detected. I don't think there's an easy solution to this other than fine-tuning (?) the model to be 'half-precision-aware' (sort of like mixed-precision training).