SuperPoint inference tooks around 8s
goktugyildirim4d opened this issue · 1 comments
I exported the SuperPoint model with the maximum keypoint count of 1024. Then, I integrated this model for a SLAM system. For some images, inference takes around 6-8s with this warning and gives 1024 keypoint as an output:
2023-08-07 14:19:07.972980870 [W:onnxruntime:Default, tensorrt_execution_provider.h:75 log] [2023-08-07 11:19:07 WARNING] [RemoveDeadLayers] Input Tensor /Gather_3_output_0 is unused or used only at compile-time, but is not being removed.
2023-08-07 14:19:16.578311157 [W:onnxruntime:Default, tensorrt_execution_provider.h:75 log] [2023-08-07 11:19:16 WARNING] TensorRT encountered issues when converting weights between types and that could affect accuracy.
2023-08-07 14:19:16.578341316 [W:onnxruntime:Default, tensorrt_execution_provider.h:75 log] [2023-08-07 11:19:16 WARNING] If this is not the desired behavior, please modify the weights or retrain with regularization to adjust the magnitude of the weights.
2023-08-07 14:19:16.578347546 [W:onnxruntime:Default, tensorrt_execution_provider.h:75 log] [2023-08-07 11:19:16 WARNING] Check verbose logs for the list of affected weights.
2023-08-07 14:19:16.578354393 [W:onnxruntime:Default, tensorrt_execution_provider.h:75 log] [2023-08-07 11:19:16 WARNING] - 2 weights are affected by this issue: Detected subnormal FP16 values.
2023-08-07 14:19:16.578362269 [W:onnxruntime:Default, tensorrt_execution_provider.h:75 log] [2023-08-07 11:19:16 WARNING] - 2 weights are affected by this issue: Detected values less than smallest positive FP16 subnormal value and converted them to the FP16 minimum subnormalized value.
2023-08-07 14:19:16.578378119 [W:onnxruntime:Default, tensorrt_execution_provider.h:75 log] [2023-08-07 11:19:16 WARNING] - 2 weights are affected by this issue: Detected finite FP32 values which would overflow in FP16 and converted them to the closest finite FP16 value.
any idea?
Also, in the case of exporting the model with the maximum keypoint count of 512. It gives the same warning with 512 keypoint as an output.
Just to be clear, is the measured 6-8s inference time only the SuperPoint extractor, or does it also include LightGlue matching time? Could you provide the image files that result in abnormal inference times so that I can try to reproduce this?
As for the warnings, the TensorRT Execution Provider is rather unpredictable right now. I would recommend using the CUDA Execution Provider for stability and more consistent inference latencies.