Multithreaded execution error.
Closed this issue · 1 comments
Good afternoon. Multithreaded handling of DetectAsync and Detect in GPU mode causes an error and detection no longer works.
2023-09-24 20:39:41.0146901 [E:onnxruntime:CSharpOnnxRuntime, cuda_call.cc:116 onnxruntime::CudaCall] CUDA failure 700: an illegal memory access was encountered ; GPU=0 ; hostname=DESKTOP-39DKFBQ ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\gpu_data_transfer.cc ; line=63 ; expr=cudaMemcpyAsync(dst_data, src_data, bytes, cudaMemcpyHostToDevice, static_cast<cudaStream_t>(stream.GetHandle())); 2023-09-24 20:39:41.0256717 [E:onnxruntime:CSharpOnnxRuntime, cuda_call.cc:116 onnxruntime::CudaCall] CUDA failure 700: an illegal memory access was encountered ; GPU=0 ; hostname=DESKTOP-39DKFBQ ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_execution_provider.cc ; line=409 ; expr=cudaStreamSynchronize(static_cast<cudaStream_t>(stream_)); 2023-09-24 20:39:41.0299946 [E:onnxruntime:CSharpOnnxRuntime, cuda_call.cc:116 onnxruntime::CudaCall] CUDA failure 700: an illegal memory access was encountered ; GPU=0 ; hostname=DESKTOP-39DKFBQ ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\gpu_data_tError: Microsoft.ML.OnnxRuntime.OnnxRuntimeException: [ErrorCode:Fail] CUDA failure 700: an illegal memory access was encountered ; GPU=0 ; hostname=DESKTOP-39DKFBQ ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\gpu_data_transfer.cc ; line=63 ; exprransfer.cc ; line=63 ; expr=c=cudaMemcpyAsync(dst_data, src_data, bytes, cudaMemcpyHostToDevice, static_cast<cudaStream_t>(stream.GetHandle())); at Microsoft.ML.OnnxRuntime.NativeApiStatus.VerifySuccess(IntPtr nativeStatus) at Microsoft.ML.OnnxRuntime.InferenceSession.RunImpluda(RunOptions options, IntPtr[] inputNames, IntPtr[] inputValues, IntPtr[] outputNames, DisposableList1 cleanupList)
at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection1 inputs, IReadOnlyCollection1 outputNames, RunOptions options)
at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection1 inputs, IReadOnlyCollection1 outputNames)
at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection1 inputs) at Compunet.YoloV8.YoloV8.Run[TResult](ImageSelector selector, PostprocessContext1 postprocess)
at Compunet.YoloV8.YoloV8Extensions.Detect(YoloV8 predictor, ImageSelector selector)
at Compunet.YoloV8.YoloV8Extensions.<>c__DisplayClass5_0.b__0()
at System.Threading.Tasks.Task1.InnerMemcpyAsync(dst_data, src_data, bytes, Invoke() at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location --- at System.Threading.EcudaMemcpyHostToDevice, static_cast<cudaStreaxecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread) --- End of stack trace from previous location --- at Compunet.YoloV8.YoloV8Extensions.DetectAsync(YoloV8 predictor, ImageSelector selector) at StressTest.DetectService.DetectAsync(String base64Img) in D:\Shared\sharpprogects\VisionSystems\StressTest\DetectService.cs:line 50 at StressTest.Program.<>c__DisplayClass1_0.<<RunTest>b__3>d.MoveNext() in D:\Shared\sharpprogects\VisionSystems\StressTest\Program.cs:line 91 m_t>(stream.GetHandle())); 2023-09-24 20:39:41.0649941 [E:onnxruntime:CSharpOnnxRuntime, cuda_call.cc:116 onnxruntime::CudaCall] CUDA failure 700: an illegal memory access was encountered ; GPU=0 ; hostname=DESKTOP-39DKFBQ ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_execution_provider.cc ; line=409 ; expr=cudaStreamSynchronize(static_cast<cudaStream_t>(stream_));
It seems that ONNX-Runtime is unable to handle Multithreading in GPU mode, so it's not a problem with this repo.
Try opening an issue about it here: https://github.com/microsoft/onnxruntime