fabio-sim/LightGlue-ONNX

some problem that delopy on device, such as snapdragon snpe.

Closed this issue · 1 comments

I want to deploy lightglue on the mobile device(snapdragon snpe), but snapdragon snpe requires a fixed dimension of input. How can the lightgule ensure effectiveness?

Hi @guker, thank you for your interest in LightGlue-ONNX.

I'm not an expert on snapdragon devices, but to answer your question on static inputs, it is in general quite tricky because LightGlue is dynamic in both inputs and outputs, which particularly have not just shape-dependent, but data-dependent dynamism.

One way to enforce a static input shape for the keypoints is to have the feature extractor output a fixed number of keypoints, either by padding or truncating to, e.g., the top-1024 keypoints with the highest confidence scores. However, ensuring that the output shapes are static is a different story, since the number of matches that are actually detected by LightGlue will not be known in advance and thus it's difficult to preallocate the output buffer exactly. A workaround that others have tried is to offload this match detection/postprocessing to outside the model (see #59).