PyTorch+ROCm for AMD Linux maybe?

Question

PyTorch+ROCm for AMD Linux maybe?

reaperx7 opened this issue 8 months ago · 13 comments

Is there anyway we can get PyTorch with ROCm support added for AMD GPUs as an experimental alternative to Tensor-RT to help reduce CPU usage for AMD GPUs?

Many thanks.

umireon commented 8 months ago

@royshil

Answer 1 · 2024-01-23T21:02:39.000Z

rocm in general is supported by ONNX runtime: https://onnxruntime.ai/docs/execution-providers/ROCm-ExecutionProvider.html
i just don't have a solid way of testing this
also we would need to build onnxruntime with rocm support: https://onnxruntime.ai/docs/build/eps.html#amd-rocm
since the linux release for ORT doesn't include the rocm EP

only CUDA and TRT
so i'm not sure... we could certainly use some help w this

Answer 2 · 2024-01-27T14:43:57.000Z

@royshil We can provide build instructions with ROCm rather than provide pre-built binaries.

@reaperx7 Are you so kind as to develop a ROCm-enabled version of our plugin? We will appreciate your contributions!

Answer 3 · 2024-01-27T14:50:15.000Z

I would if I could, but I can't read code worth a penny.

Answer 4 · 2024-01-27T15:02:33.000Z

@reaperx7 Do you have an environment to test our ROCm-enabled plugin if we provide you with build instructions?

Answer 5 · 2024-01-27T22:51:21.000Z

I have an up to date ArchLinux system with the ROCm libraries, amdgpu-pro installed, and an RX 5700XT. If it's buildable, and all I need to do is drag and drop it into the plugin folder for OBS, then... Yes. Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Kaito Udagawa ***@***.***> Sent: Saturday, January 27, 2024 7:02:44 AM To: occ-ai/obs-backgroundremoval ***@***.***> Cc: James Powell ***@***.***>; Mention ***@***.***> Subject: Re: [occ-ai/obs-backgroundremoval] PyTorch+ROCm for AMD Linux maybe? (Issue #535) @reaperx7<https://github.com/reaperx7> Do you have an environment to test our ROCm-enabled plugin if we provide you with build instructions? — Reply to this email directly, view it on GitHub<#535 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACPXB6SHLL4RGKOXGWN7RV3YQUJJJAVCNFSM6AAAAABCHAAPSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTGE4DAOBQHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 6 · 2024-02-08T03:54:15.000Z

Just dropping in to confirm I was able to build the plugin using the C/C++ API of the ROCm Execution Provider. All I needed to do was to modify ort-session-utils.cpp to add the ROCm EP to the ONNX Runtime session instead of the TensorRT EP, and to then to set the appropriate flag in CMake to use the system's own ONNX Runtime presumably built for ROCm.

Answer 7 · 2024-02-09T08:05:51.000Z

@reaperx7 @payom I have implemented the initial ROCm support. Can you test this?
#545

Answer 8 · 2024-02-09T09:47:22.000Z

I can try and test it if a tarball is available. Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Kaito Udagawa ***@***.***> Sent: Friday, February 9, 2024 12:06:03 AM To: occ-ai/obs-backgroundremoval ***@***.***> Cc: James Powell ***@***.***>; Mention ***@***.***> Subject: Re: [occ-ai/obs-backgroundremoval] PyTorch+ROCm for AMD Linux maybe? (Issue #535) @reaperx7<https://github.com/reaperx7> @payom<https://github.com/payom> I have implemented the initial ROCm support. Can you test this? #545<#545> — Reply to this email directly, view it on GitHub<#535 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACPXB6VBBOI5PD4UXITLOELYSXKGXAVCNFSM6AAAAABCHAAPSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZVGQ4DQNZVG4>. You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 9 · 2024-02-09T12:18:04.000Z

@reaperx7 You must build the binary by yourself. We will not provide any pre-built things. Thank you!

Answer 10 · 2024-02-10T05:52:35.000Z

That's fine. I just need the sources either in tarball or the exact git repository. Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Kaito Udagawa ***@***.***> Sent: Friday, February 9, 2024 4:18:16 AM To: occ-ai/obs-backgroundremoval ***@***.***> Cc: James Powell ***@***.***>; Mention ***@***.***> Subject: Re: [occ-ai/obs-backgroundremoval] PyTorch+ROCm for AMD Linux maybe? (Issue #535) @reaperx7<https://github.com/reaperx7> You must build the binary by yourself. We will not provide any pre-built things. — Reply to this email directly, view it on GitHub<#535 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACPXB6TG7HWWFAFJL7ICHY3YSYHYRAVCNFSM6AAAAABCHAAPSCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZVHAZDQNRVHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 11 · 2024-02-12T01:33:51.000Z

Tried enabling plugin and got error and obs crashed. log file attached.
error.log

Source built with flags:

cmake -B build --preset linux-x86_64 -DENABLE_ROCM=ON

Answer 12 · 2024-04-12T23:31:18.000Z

This issue is stale because it has been open for 60 days with no activity.