NOT_IMPLEMENTED : Non-zero status code returned while running MultiHeadAttention node
dmoti opened this issue · 2 comments
I'm using the fused version of the models, it runs ok for several images until I'm getting this error:
[ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Non-zero status code returned while running MultiHeadAttention node. Name:'MultiHeadAttention_1' Status Message: packed QKV format is not implemented for current GPU. Please disable it in fusion options.
File "/data/eng/users/motid/work/findle-rel/findle-query/models/LightGlueONNX/onnx_runner/lightglue.py", line 53, in run
matches0, mscores0 = self.lightglue.run(
File "/data/eng/users/motid/work/findle-rel/findle-query/rerank/light_glue_reranker.py", line 125, in onnx_matcher_pair
return self.runner.run(image0, image1, scales0, scales1)
File "/data/eng/users/motid/work/findle-rel/findle-query/rerank/light_glue_reranker.py", line 153, in calculate_inliers_count
kpts_query, kpts_pred, conf = self.onnx_matcher_pair(q_path, c_path)
File "/data/eng/users/motid/work/findle-rel/findle-query/rerank/reranker.py", line 70, in rerank_by_matcher
self.res_dict, time_mean = self.calculate_inliers_count(queries, database)
File "/data/eng/users/motid/work/findle-rel/findle-query/algo_iface/FindleAlgo.py", line 31, in rerank
return self.__reranker.rerank_by_matcher([query], preds_array)
File "/data/eng/users/motid/work/findle-rel/findle-query/findle_query/clients/vespa_client.py", line 113, in query
res_dict, rerank_time_mean = findle_algo.rerank(image_filename, merged_paths)
File "/data/eng/users/motid/work/findle-rel/findle-query/findle_query/services/findle_query_service.py", line 34, in query
predictions = self.__vespa_client.query(self.findle_algo, image_filename, self.__crop_query, dump_output_folder=self.__dump_output_folder)
File "/data/eng/users/motid/work/findle-rel/findle-query/findle_query/api/routers/findle_query_router.py", line 24, in query
result = findle_query_service.query(request.path)
onnxruntime.capi.onnxruntime_pybind11_state.NotImplemented: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Non-zero status code returned while running MultiHeadAttention node. Name:'MultiHeadAttention_1' Status Message: packed QKV format is not implemented for current GPU. Please disable it in fusion options.
Please tell me how to solve it
Hi @XL634663985, thank you for your interest in LightGlue-ONNX.
Regarding the packed QKV error for MultiheadAttention node, you can try using the fused_cpu versions in https://github.com/fabio-sim/LightGlue-ONNX/releases/tag/v1.0.0 . The only difference is that these use unpacked QKV, so they can also run on GPU.