Using GPU
Desync3 opened this issue · 13 comments
How can i use the gpu instead of cpu?
I would like to know the same. Updated the nuget package to OnnxRuntime.GPU
According to https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.transforms.onnx.onnxscoringestimator?view=ml-dotnet for Cuda support you need to set gpuDeviceId when creating the estimator using ApplyOnnxModel. But where is that?
scorer = new YoloScorer("Assets/Weights/yolov5n.onnx" ,SessionOptions.MakeSessionOptionWithCudaProvider());
I try this way,but failed,
error msg =Attempt to use DefaultLogger but none has been registered.
has anyone success use GPU?
scorer = new YoloScorer("Assets/Weights/yolov5n.onnx" ,SessionOptions.MakeSessionOptionWithCudaProvider());
I try this way,but failed,
error msg =Attempt to use DefaultLogger but none has been registered.
has anyone success use GPU?
Check this thread. Might solve it. You need to register de logger before make session. I have code for you. Fix on Monday latest, else give me a shout
scorer = new YoloScorer("Assets/Weights/yolov5n.onnx" ,SessionOptions.MakeSessionOptionWithCudaProvider());
I try this way,but failed,
error msg =Attempt to use DefaultLogger but none has been registered.
has anyone success use GPU?Check this thread. Might solve it. You need to register de logger before make session. I have code for you. Fix on Monday latest, else give me a shout
do u success with cuda?how may fps? i find python yolov5 gpu cost losts memory than cpu.
does OnnxRuntime.GPU use memory a lot?
Yes I can run GPU support with Cuda. The FPS has not been evaluated fully but that depends on
size of the Yolo model. You have small, medium large. Also the size of the input image.
you also need to have a GPU big enough. I have only a Quadro M1200 so not so strong.
The code I had did not have multi-threading so I only used maybe 30% of the GPU. So a multithreaded version is needed.
You need to try this on your GPU and images.
scorer = new YoloScorer("Assets/Weights/yolov5n.onnx" ,SessionOptions.MakeSessionOptionWithCudaProvider());
I try this way,but failed,
error msg =Attempt to use DefaultLogger but none has been registered.
has anyone success use GPU?Check this thread. Might solve it. You need to register de logger before make session. I have code for you. Fix on Monday latest, else give me a shout
microsoft/onnxruntime#10336do u success with cuda?how may fps? i find python yolov5 gpu cost losts memory than cpu.
does OnnxRuntime.GPU use memory a lot?
Did you manage to get it working?
I added SessionOptions.MakeSessionOptionWithCudaProvider() but the speed is still the same as the CPU, I checked task manager and GPU usage is 5% and sometimes even 0%, does this really utilize the GPU?
If smb still looking for solution. Looks like I have one.
CUDA version: cuda_11.4.4_472.50_windows
cuDNN version: cudnn-11.4-windows-x64-v8.2.2.26.zip (all dll placed into project output dir)
nuget: Microsoft.ML.OnnxRuntime.Gpu 1.13.0
Set options with sample code (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#install):
var cudaProviderOptions = new OrtCUDAProviderOptions(); // Dispose this finally
var providerOptionsDict = new Dictionary<string, string>();
providerOptionsDict["device_id"] = "0";
providerOptionsDict["gpu_mem_limit"] = "2147483648";
providerOptionsDict["arena_extend_strategy"] = "kSameAsRequested";
providerOptionsDict["cudnn_conv_algo_search"] = "DEFAULT";
providerOptionsDict["do_copy_in_default_stream"] = "1";
providerOptionsDict["cudnn_conv_use_max_workspace"] = "1";
providerOptionsDict["cudnn_conv1d_pad_to_nc1d"] = "1";
cudaProviderOptions.UpdateOptions(providerOptionsDict);
SessionOptions options = SessionOptions.MakeSessionOptionWithCudaProvider(cudaProviderOptions); // Dispose this finally
Use it like this:
using (var scorer = new YoloScorer("Assets/Weights/yolov5s.onnx", options))
If smb still looking for solution. Looks like I have one. CUDA version: cuda_11.4.4_472.50_windows cuDNN version: cudnn-11.4-windows-x64-v8.2.2.26.zip (all dll placed into project output dir) nuget: Microsoft.ML.OnnxRuntime.Gpu 1.13.0
Set options with sample code (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#install): var cudaProviderOptions = new OrtCUDAProviderOptions(); // Dispose this finally
var providerOptionsDict = new Dictionary<string, string>(); providerOptionsDict["device_id"] = "0"; providerOptionsDict["gpu_mem_limit"] = "2147483648"; providerOptionsDict["arena_extend_strategy"] = "kSameAsRequested"; providerOptionsDict["cudnn_conv_algo_search"] = "DEFAULT"; providerOptionsDict["do_copy_in_default_stream"] = "1"; providerOptionsDict["cudnn_conv_use_max_workspace"] = "1"; providerOptionsDict["cudnn_conv1d_pad_to_nc1d"] = "1";
cudaProviderOptions.UpdateOptions(providerOptionsDict);
SessionOptions options = SessionOptions.MakeSessionOptionWithCudaProvider(cudaProviderOptions); // Dispose this finally
Use it like this: using (var scorer = new YoloScorer("Assets/Weights/yolov5s.onnx", options))
After 2 years I've decided to continue to work on my project to get yolo v5 to work with gpu.
I have rtx 3070 ti, and my python yolo v5 project works but i wanna do c#.
I have installed cuda and cudnn, I also have the microsoft.ml.onnxruntime.gpu 1.14.1. I'm stuck and i have no idea how to fix this. I just get error when i try to use gpu.
The error:
Microsoft.ML.OnnxRuntime.OnnxRuntimeException: '[ErrorCode:RuntimeException] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1106 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\water\source\repos\yolov5-net-master\src\Yolov5Net.App\bin\Debug\net6.0\runtimes\win-x64\native\onnxruntime_providers_cuda.dll"
'
Please help, thanks.
Just some common idea. Set your project to 64bit and be sure all dlls like cudnn_*** from cuDNN and onnxruntime_*** placed around your exe. Or better check: x64\Debug\net7.0-windows10.0.22621.0\runtimes\win-x64\native\ and add cudnn dlls there.
After 2 years I've decided to continue to work on my project to get yolo v5 to work with gpu. I have rtx 3070 ti, and my python yolo v5 project works but i wanna do c#.
I have installed cuda and cudnn, I also have the microsoft.ml.onnxruntime.gpu 1.14.1. I'm stuck and i have no idea how to fix this. I just get error when i try to use gpu.
The error: Microsoft.ML.OnnxRuntime.OnnxRuntimeException: '[ErrorCode:RuntimeException] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1106 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\water\source\repos\yolov5-net-master\src\Yolov5Net.App\bin\Debug\net6.0\runtimes\win-x64\native\onnxruntime_providers_cuda.dll" '
Please help, thanks.
1.CUDA版本11.2
2.cuDNN用cudnn-windows-x86_64-8.9.3.28_cuda11-archive
记得把压缩包的三个文件夹放到cuda根目录下覆盖
3.Microsoft.ML.OnnxRuntime.Gpu要使用1.13.1,如果版本太新，SessionOptions会报错。
————————————————
版权声明：本文为CSDN博主「xcagy」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/ccagy/article/details/133847381
After 2 years I've decided to continue to work on my project to get yolo v5 to work with gpu. I have rtx 3070 ti, and my python yolo v5 project works but i wanna do c#.
I have installed cuda and cudnn, I also have the microsoft.ml.onnxruntime.gpu 1.14.1. I'm stuck and i have no idea how to fix this. I just get error when i try to use gpu.
Please help, thanks.1.CUDA版本11.2
2.cuDNN用cudnn-windows-x86_64-8.9.3.28_cuda11-archive
记得把压缩包的三个文件夹放到cuda根目录下覆盖
3.Microsoft.ML.OnnxRuntime.Gpu要使用1.13.1,如果版本太新，SessionOptions会报错。 ———————————————— 版权声明：本文为CSDN博主「xcagy」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。 原文链接：https://blog.csdn.net/ccagy/article/details/133847381
Did you follow this:
microsoft/onnxruntime#10336