No detection when using GPU, but CPU works

[acschristoph]

Hey,

When I run detect_video.py with no modifications there are no detections.
when I force tensorflow to use cpu instead of gpu I got detections.

Does anybody have maybe a hint for me, I tried diffrent version of tf with same result.
Thanks

tensorflow tested: 2.5, 2.6, 2.7
Py: 3.9.7
Cuda: 11.5
Cudnn 8.3.2.44
GPU: NVIDIA GeForce RTX 2060
OS: Win10

Output
F:\code projects\ocn>python detect_video.py 2022-02-05 09:47:20.211848: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll 2022-02-05 09:47:23.408474: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library nvcuda.dll 2022-02-05 09:47:23.436653: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: pciBusID: 0000:2d:00.0 name: NVIDIA GeForce RTX 2060 computeCapability: 7.5 coreClock: 1.68GHz coreCount: 30 deviceMemorySize: 6.00GiB deviceMemoryBandwidth: 312.97GiB/s 2022-02-05 09:47:23.444030: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll 2022-02-05 09:47:23.462679: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublas64_11.dll 2022-02-05 09:47:23.466247: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublasLt64_11.dll 2022-02-05 09:47:23.476346: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cufft64_10.dll 2022-02-05 09:47:23.481142: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library curand64_10.dll 2022-02-05 09:47:23.499409: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusolver64_11.dll 2022-02-05 09:47:23.526262: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusparse64_11.dll 2022-02-05 09:47:23.530584: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudnn64_8.dll 2022-02-05 09:47:23.534337: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0 2022-02-05 09:47:23.546495: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2 To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-02-05 09:47:23.556502: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: pciBusID: 0000:2d:00.0 name: NVIDIA GeForce RTX 2060 computeCapability: 7.5 coreClock: 1.68GHz coreCount: 30 deviceMemorySize: 6.00GiB deviceMemoryBandwidth: 312.97GiB/s 2022-02-05 09:47:23.564094: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0 2022-02-05 09:47:24.064582: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix: 2022-02-05 09:47:24.068158: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264] 0 2022-02-05 09:47:24.070268: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0: N 2022-02-05 09:47:24.072549: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3832 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2060, pci bus id: 0000:2d:00.0, compute capability: 7.5) I0205 09:47:26.387739 13700 detect_video.py:32] weights loaded I0205 09:47:26.388740 13700 detect_video.py:35] classes loaded 2022-02-05 09:47:26.444565: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2) 2022-02-05 09:47:27.639137: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudnn64_8.dll 2022-02-05 09:47:28.442090: I tensorflow/stream_executor/cuda/cuda_dnn.cc:359] Loaded cuDNN version 8302 2022-02-05 09:47:29.228065: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublas64_11.dll 2022-02-05 09:47:29.232000: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublasLt64_11.dll 2022-02-05 09:47:29.751691: W tensorflow/core/common_runtime/bfc_allocator.cc:337] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature.