[Feature Request] GPU acceleration

[axeldelafosse]

Following this discussion: #8 (comment) -- I think Stemgen should:

• Default to GPU
• Provide a flag to force CPU
• Explain how to get Spleeter and/or Demucs to work with Mac GPUs

[0vm]

Don't default to GPU as many people will be running this on dockers, VM's, etc. These don't have GPUs and if anyone wants to switch to GPU, include something in the README such as removing the CPU Flag.

[awesomer]

I think it's fine to default to GPU... *if* the underlying tool automatically and reliably falls back to CPU if GPU is not available. Otherwise, I agree that CPU should be the default.

…

On Mon, Dec 19, 2022 at 7:34 PM Ak ***@***.***> wrote: Don't default to GPU as many people will be running this on dockers, VM's, etc. These don't have GPUs and if anyone wants to switch to GPU, include something in the README such as removing the CPU Flag. — Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADRNXCPTDO6BST3IIV36CLWOESMBANCNFSM6AAAAAATD5MCE4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[0vm]

I disagree as what if the GPU has very bad performance, shared memory with RAM, etc. An option in the README will be much more convenient as people who are interested in using the GPU will use the GPU while CPU option works on every device.

…

On Tue, 20 Dec 2022 at 15:14, awesomer ***@***.***> wrote: I think it's fine to default to GPU... *if* the underlying tool automatically and reliably falls back to CPU if GPU is not available. Otherwise, I agree that CPU should be the default. On Mon, Dec 19, 2022 at 7:34 PM Ak ***@***.***> wrote: > Don't default to GPU as many people will be running this on dockers, VM's, > etc. These don't have GPUs and if anyone wants to switch to GPU, include > something in the README such as removing the CPU Flag. > > — > Reply to this email directly, view it on GitHub > < #9 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AADRNXCPTDO6BST3IIV36CLWOESMBANCNFSM6AAAAAATD5MCE4 > > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > — Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATBSFS2JKABRW37S2YQPYD3WOEXDVANCNFSM6AAAAAATD5MCE4> . You are receiving this because you commented.Message ID: ***@***.***>

[awesomer]

There are cases where GPU acceleration has worse performance than CPU?

[benjaminlewandowski]

There are cases where GPU acceleration has worse performance than CPU?

I think that some gpu/cpu combinations could be slower with cuda - but it should be the exception

Using gpu is around 15 times faster with my mobile 3070 instead of my 10th gen 16 thread I7.

I will also test this with a 11th gen i5/ mobile 1650 gpu combo

Update:
According to userbenchmarks a 10 year old gtx 660 is a 5th as fast (or rather slow) as the 3070 mobile - so even that card should be much faster then running on cpu

[axeldelafosse]

Yeah! I'm gonna work on this soon!

[MacTheZazou]

It's quite easy to use GPU with stemgen:

1. Follow the GPU step of DEMUCS setup:
   If you are using Windows: Windows support.
   If you are using MAC OS X: Mac OS X support.
   If you are using Linux: Linux support.

2. replace all occurence of
   "-d", "cpu",
   for
   "-d", "cuda",

in stemgen.py

[axeldelafosse]

Hey! I took some time to write proper docs and add support for GPU without touching the code: https://stemgen.dev/docs/gpu-acceleration-on-windows.

Note that it is only working on Windows if you have a NVIDIA GPU.