not using gpu

Question

not using gpu

Closed this issue a month ago · 30 comments

Hi, it's using 100% of my cpu and 0% of gpu all the time, memory is 0.0gb too

Answer 1 · 2024-05-15T19:44:43.000Z

What version of python ate you using? This issue can occur when using an outdated python version. I know that Python 3.11.4 works for certain. 3.10 does not work.

Answer 2 · 2024-05-15T22:50:03.000Z

i have python 3.11.4

Answer 3 · 2024-05-16T09:07:49.000Z

Hi @xxopenxx.

Are you on Windows 10 or Windows 11? because onnxruntime-directml (the AI engine) is only compatible with this OS.

Otherwhise it will use the CPU

Answer 4 · 2024-05-16T18:15:33.000Z

Hi @xxopenxx.

Are you on Windows 10 or Windows 11? because onnxruntime-directml (the AI engine) is only compatible with this OS.

Otherwhise it will use the CPU

WINDOWS 10

Answer 5 · 2024-05-16T18:16:00.000Z

Python 3.10.14, I never had problem with any ai open source apps, like stable diffusion, ollama etc

Answer 6 · 2024-05-17T04:10:36.000Z

@xxopenxx can you send your PC specs? (Task manager / performance tab)

Answer 7 · 2024-05-17T13:19:17.000Z

i5 12400f, rtx 4070 12gb vram, 16gb ram

Answer 8 · 2024-05-17T13:43:45.000Z

Have you installed Visual C Redistributable and DXRuntime?

(If not already installed) install Visual-C-Runtimes:
- https://www.techpowerup.com/download/visual-c-redistributable-runtime-package-all-in-one/
(If not already installed) install Directx:
- https://www.microsoft.com/it-it/download/details.aspx?id=35

typically should already be installed

Answer 9 · 2024-05-17T18:50:59.000Z

These 2 things already installed

Answer 10 · 2024-05-17T19:13:24.000Z

Strange, are you sure it is not using your gpu? the temperature is "normal" while upscaling?

Also, if you are on Python, you should see an error that says "onnxruntime will use CPU" in the terminal

Answer 11 · 2024-05-17T19:24:58.000Z

i dont have that error, there are no errors at all in terminal of quality scaler

Answer 12 · 2024-05-17T19:26:58.000Z

Are you selecting "High performance" GPU in the options right? and if you select Power saving?

Answer 13 · 2024-05-18T00:14:12.000Z

Python 3.10.14, I never had problem with any ai open source apps, like stable diffusion, ollama etc

wait.... do you have python 3.10.14 or 3.11.4? Because this looks an awfully lot like this issue #69

Answer 14 · 2024-05-18T15:31:44.000Z

xxopenxx commented 4 months ago

Answer 15 · 2024-05-18T15:37:17.000Z

xxopenxx commented 4 months ago

Answer 16 · 2024-05-18T15:44:58.000Z

Hi my friend, maibe you do not see the correct "graph". In fact, you are seeing only "Video encode" e "Video decode", but there are more options. To see if the gpu is being used, you have to switch to "3D" graph.

Also, you can use VsCode to run QualityScaler.py and you should see all logs

Answer 17 · 2024-05-18T15:47:05.000Z

But while im using stable diffusion and im upscaling photo for example, I have 99% gpu usage in task manager, it's same for ollama and video diffusion. Upscaling using qualityscaler takes 2h to upscale 2minutes video from 360 p to 760p, using multihreading

Answer 18 · 2024-05-18T15:48:50.000Z

sorry but, just a stupid question, have you installed the correct dependencies in requirements.txt?

Answer 19 · 2024-05-18T15:51:11.000Z

xxopenxx commented 4 months ago

Answer 20 · 2024-05-18T15:51:55.000Z

i can share screen on discord or something like that if you want

Answer 21 · 2024-05-18T15:54:51.000Z

can you unistall everything and re-install? i think there is some problem somewere because the latest version of onnxruntime-directml is 1.17.3, but you have 1.17.0

pip freeze > requirements.txt
pip uninstall -r requirements.txt -y
pip cache purge

Answer 22 · 2024-06-06T10:37:05.000Z

@Djdefrag Can I run it if I don't have GPU ?
If so how long would it take to process lets say a 5 min video on a 8GM RAM (I5 3rd gen) machine ?

Answer 23 · 2024-06-07T07:17:53.000Z

Hi my friend,

yes, it will run without GPU, but will be really slow.

Maibe you can try using the lightest AI model, RealSR_Gx4

Answer 24 · 2024-06-18T23:25:22.000Z

Hello - I have the same issue. From both steam app and version 3.6 from github. The symptom is that CPU is peak and no GPU usage (no memory allocation, CUDA is low % (e.g. 0-5% same as GPU).

I found this on stack and tried the script.
https://stackoverflow.com/questions/64452013/how-do-you-run-a-onnx-model-on-a-gpu

Mine say, "CPU-DML". Is that correct?

I also check python package and it is using

onnxconverter-common==1.14.0     
onnxruntime-directml==1.18.0

Answer 25 · 2024-06-19T01:02:04.000Z

ok - some update. installed onnxruntime-gpu and onnxruntime (cpu) and tried to use it. It turned out that estimates say 19hrs+ rather than 30mins with onnxruntime-directml.

Then, I look at GPU temperature, and temp is a bit higher than usual (70C). So, I guess, task manager may not report the right GPU metrics?

Answer 26 · 2024-06-19T06:08:24.000Z

ok - some update. installed onnxruntime-gpu and onnxruntime (cpu) and tried to use it. It turned out that estimates say 19hrs+ rather than 30mins with onnxruntime-directml.

Then, I look at GPU temperature, and temp is a bit higher than usual (70C). So, I guess, task manager may not report the right GPU metrics?

Hi my friend, yes, with Nvidia GPU Task manager does not visualize "AI processing" by default. You have to select the correct Voice from the graph menu, probably Compute 0 or High priority 3D

Answer 27 · 2024-06-20T21:25:26.000Z

I couldn't figure out where the bottleneck is. It doesn't use full - cpu or GPU. VRAM is very low as well.

(note: I added hvec_nvenc for faster hardware encoding, using it with P7 still several times faster and I couldn't tell the different between software/hardware for my use case)

Answer 28 · 2024-06-22T10:05:40.000Z

Hi my friend,

usually Half precision is faster, i reccomend use Full precision only for old gpus (which are not compatible with Half precision).

For Nvidia gpu's, Task Manager does not show GPU usage for AI processing by default. It should show it in one of these graphs, but I don't know which one because I don't have an Nvidia GPU :(

In any case, it is also useful to look at the GPU temperature, to see if it is being used. If the temperature is high, it means it is being used properly.

Answer 29 · 2024-06-23T18:52:14.000Z

For me, it shows as Graphic_1 above which is running around 67%. I don't have High priority 3D.
Yeh. I have been playing around with Half precision setting, I settled with half precision.

Do you have recommendation on interpolation? I noticed improvement selecting "None". However, I don't know if it's happening in every frame.

btw, I solved the GPU/CPU not fully used problem. I made change to allow 6 threads, now GPU goes up around 80%, I think VRAM is 1GB more and CPU is bounded :). It's reduced1.5 hours of 5 hrs processing with 4 threads. I think Real-ESRGAN can use high # of thread setting. But ESRGAN, will burn out my CPU much quicker.

Thanks for the note on watching CPU/GPU temp on the app, that's how I found out.

Answer 30 · 2024-08-13T09:00:34.000Z

I was having the same issue but I worked it out thanks to this thread to help me understand the requirements.
I was ripping out my hair because I had just reinstalled a clean copy of Windows 11 23H2..but I copied over my AppData from my old install.
Turned out the permissions werent right when I attempted to install Ubuntu with WSL2 to try running it under Linux Subsystem.
After fixing my AppData permissions and checking WSL2 was working correctly; I reinstalled the requirements using pip..
Went from 100% CPU and 0% GPU with calculated 120hrs; to 20% CPU and 40% 3D on the GPU and dropped to 3hrs

Relevant Specs:
Ryzen9 5950x
Nvidia RTX3080Ti
Windows 11 Pro 23H2
All VC Redists
Python 3.12
onnxruntime-directml 1.17.3

For completeness of this discussion, heres what it looks like when its working properly with GPU in Windows