FP16 model can not get acceleration on GPU with ONNXRuntime-GPU

Question

FP16 model can not get acceleration on GPU with ONNXRuntime-GPU

yeliang2258 opened this issue 2 years ago · 2 comments

Hello,
I use the float16 tool to convert the FP32 model to the FP16 model and use ONNXRuntime-GPU 1.13.1 to inference.
I found that many models cannot obtain inference acceleration.
I want to know what kind of ONNX FP32 models can obtain inference acceleration after converting to FP16? ?
Looking forward to your answer, thank you

Answer 1 · 2023-02-15T02:16:11.000Z

@abock Hi, Please help me to see my problem, thank you

Answer 2 · 2024-03-25T23:14:31.000Z

Hi @yeliang2258 ,

We’ve gone ahead and closed this issue because it is stale.
If you still believe this issue is relevant, please feel free to reopen the issue and we will triage it as necessary.

Apologies for not addressing it at the time it was opened.