Integration to kornia?
ducha-aiki opened this issue · 6 comments
Hi,
Would you be interested in integrating LightGlue ONNX into https://github.com/kornia/kornia?
--
Best, Dmytro
Hi @ducha-aiki, thank you for your interest in LightGlue-ONNX.
I would be glad to add LightGlue-ONNX to kornia, however, my understanding is that kornia is based on PyTorch and does not have ONNXRuntime as a dependency. Could you point me to an example/PR of another ONNX model that has been added to kornia?
You are corrent, however recenty we decided to go with optional dependencies for extra features and onnx is one of them. We don't have an example yet, but me and Edgar could give you a hand with this.
@fabio-sim i believe this is a good use case to start adding onnxruntime as dep. I would say the right approach could be o update https://github.com/kornia/kornia/blob/master/kornia/feature/lightglue.py according with what's needed from your repo ?
Hi @edgarriba, @ducha-aiki, thanks for the messages.
Could you elaborate on what specifically you would like to integrate to kornia? (E.g., should users be able to export to ONNX using kornia, or would an auto-downloader for the ONNX weights be more appropriate? In the latter case, kornia will only need to add a thin wrapper for the ONNXRuntime inference runner)
@edgarriba I would advise against updating the PyTorch LightGlue model code, because the modifications are not compatible and would change the behaviour of the class (in particular the way Attention modules are handled).
I think that downloading runtime model weights is what we are after. And for the conversion people can use this repo (which we will put the link in the documentation)
I understand. Thanks for the response.
I will try to contribute a PR :)