Install Reshade for a Vulkan game

Question

Install Reshade for a Vulkan game

rylydou opened this issue 3 years ago · 63 comments

I am installing Reshade into Cemu which uses Vulkan. I know that on Windows you can install Reshade on a Vulkan game. But I don't know what dll name to use or if it is even possible.

I have tried:

dxgi - The suggested option
opengl32 - Because Cemu has an option to switch to that but I would prefer to use Vulkan for performance
vulkan

Answer 1 · 2022-01-16T16:55:25.000Z

I'm not sure, never used Cemu.

While doing a quick search I found this: https://i.imgur.com/o8nwDkd.jpg linked here https://teddit.net/r/CemuPiracy/comments/ey6ien/is_there_any_tutorial_for_reshade_on_cemu_botw/

Does that mean the games need patches for Reshade to function?

Looking at this video at 4m50s, there's no dll file in this Cemu folder, I'm guessing Reshade does something different to hook into games using Vulkan?

Answer 2 · 2022-01-16T17:02:39.000Z

ReShade does not support DLL wrapping in Vulkan. Vulkan has a layering mechanism that ReShade has to use to be spec conform. And that requires ReShade to be in a central place, because those layers are configured via system-wide registry keys. You are best off using the setup tool to set up ReShade, it knows what to do to make it work.

https://reshade.me/forum/troubleshooting/6153-reshade-and-vulkan

I'm guessing if we figure out which registrey keys are added, then we could try to do that programatically?

Answer 3 · 2022-01-16T17:03:59.000Z

For now, you could use vkBasalt which supports most Reshade shaders, only issue is there's no in game GUI.

Answer 4 · 2022-01-16T17:07:58.000Z

This post explains how the installer sets up the registry:

https://reshade.me/forum/troubleshooting/6153-reshade-and-vulkan#41351

Answer 5 · 2022-01-16T17:17:01.000Z

Thanks, I'll give vkBasalt with goverlay a try. Thanks for the help!

Answer 6 · 2022-01-16T23:10:54.000Z

Not tested in a game, but I've got something that might work if you want to test.

The WINEPREFIX has to be set for the game in particular and the ReShade64.json needs to be changded to ReShade32.json if the program is 32 bits, the reg:64 also has to be changed to reg:32 if it's a 32 bit prefix.

WINEPREFIX="$HOME/Games/windows/lutris/Test/wineprefix" wine reg ADD HKLM\\SOFTWARE\\Khronos\\Vulkan\\ImplicitLayers /d 0 /t REG_DWORD /v "Z:\\home\\$USER\\.reshade\\reshade\\ReShade64.json" -f /reg:64

Here's regedit in a test WINEPREFIX: https://i.imgur.com/80KiVki.png

Answer 7 · 2022-01-17T00:40:38.000Z

I've have tried doing the registry edit but I don't have either of the ReShade dlls in ~/.reshade/reshade. I do have the json files that point to them though.

Answer 8 · 2022-01-17T01:55:17.000Z

That's a good point, the dxgi.dll would have to be linked to ReShade64.dll and the d3d9.dll to ReShade32.dll

I'll add those things to the script, will have to find a Vulkan game to test on.

Answer 9 · 2022-01-17T01:58:24.000Z

Half life 2 is 32 bit and has Vulkan support : https://www.pcgamingwiki.com/wiki/Half-Life_2#API

Doom 2016 64 bit / Vulkan : https://www.pcgamingwiki.com/wiki/Doom_(2016)#API

Will test these this week and update the script.

Answer 10 · 2022-01-18T20:46:00.000Z

I've been testing on Doom, but not having any success, the registry entry is correctly added, but the game doesn't launch with Reshade.

Added the changes to a different branch:

8275417

Answer 11 · 2022-01-18T20:52:14.000Z

In a Windows VM I installed Reshade to make sure the Registry matches what's in wine:

So the issue is somewhere else.

Answer 12 · 2022-01-18T20:57:32.000Z

The issue might be with Reshade itself:

https://reshade.me/forum/suggestions/7074-linux-vulkan-opengl-version#40598

This unlikely to happen anytime soon, since large parts of the ReShade code base are Windows specific (especially the hooking code). Only thing that could potentially be separated out would be the Vulkan layer implementation, but that is already partially covered by vkBasalt so it doesn't feel worth the effort right now.

Edit: Although, (in our case) it's through Wine, but perhaps there's some Windows specific Vulkan stuff that's not available on Linux so Wine can't do anything about it?

Answer 13 · 2022-02-01T19:58:05.000Z

In ReShade 5.0, there are some changes to how the json files are handled.

Here's the relevant section:

https://github.com/crosire/reshade/blob/67f4f718614800b426f20291b2fc7d7ca6644f37/setup/MainWindow.xaml.cs#L711

Answer 14 · 2022-06-07T16:27:03.000Z

Would it be possible to install Reshade 4.9.1 with global Vulkan enabling option? This is better than nothing for now.

Answer 15 · 2022-06-10T21:09:16.000Z

Not sure, from what I can remember it doesn't function at all. Busy with work and other stuff at the moment, so not enough time to test, unfortunately.

Answer 16 · 2022-06-11T09:40:34.000Z

I tried 4.9.1, 5.0.2 and 5.2.1. None works. I tried them on Valheim and RDR 2. Valheim can also work on Dx11 so it is possible to make the installation for it but RDR 2 has only Vulkan for Linux (Dx12 freezes the PC).

I have read somewhere Wine runs all the games in administrator mode. I also know that Reshade Vulkan doesn't load when the game is run with admin privileges. Could that be the problem?

Answer 17 · 2022-08-22T22:17:03.000Z

It is not about Vulkan games but there is good news from Crosire. He fixed a problem which prevented newer Reshade versions to work with DirectX 12 games. I hope he might look into Vulkan problem as well.

Answer 18 · 2023-09-06T18:53:53.000Z

Hello, try vulkan-1.dll and let me know if worked, bye.
Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

Answer 19 · 2023-09-08T06:19:51.000Z

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

I won't have access to a Linux PC couple more days. If anyone tried it, could they please report if it works?

Answer 20 · 2023-09-13T12:08:36.000Z

Hello, try vulkan-1.dll and let me know if worked, bye. Edit: Also download the ReShade executable and extract it with 7Zip and see if is there any DLLs, if is some named ReShade32.dll and/or ReShade64.dll then rename one of them as vulkan-1.dll the JSON file is useless or it should be, otherwise if is needed just edit the JSON file and replace the ReShade(32/64).dll with vulkan-1.dll and this should work.

I won't have access to a Linux PC couple more days. If anyone tried it, could they please report if it works?

I tried this on Baldur's Gate 3 on my Linux machine where I do have Reshade working, but it did not work.

Answer 21 · 2023-09-20T06:41:32.000Z

I tried too for Borderland 2 (DX9) and TerraTech (DX11) and won't work, so sorry, it was just an idea, but works only if I run the setup and select the game executable and telling the setup is running on Vulkan, is creating "ReShade32.dll" or "ReShade64.dll" but not the JSON file in the game folder and something else is creating in "C:\ProgramData" that contain the original JSON file and maybe something in registry but I'm not able yet to track what is it or if is something added/changed. My test PC with Linux died :( sorry if I'm not able to provide more details, I just give what I remember from the Linux machine.

PS: try to run the setup of ReShade and give the game executable and select Vulkan for runtime, maybe this will work if setup takes the details correctely.

Answer 22 · 2023-12-29T12:35:07.000Z

took a while to figure out:

it works both normally and through gamescope.
really tired now. will post how to do this later :)

Answer 23 · 2023-12-29T18:29:53.000Z

This is early new year:) I am (and many more, I'm sure) looking forward to instructions.

Answer 24 · 2023-12-29T22:25:55.000Z

it works both normally and through gamescope. really tired now. will post how to do this later :)

I will gladly use those instructions! After you're well rested of course :) As I managed thanks to your preview DLL and some help from the community to have an HDR signal in System Shock 2! This might be the path towards universal HDR support on SteamOS!

Answer 25 · 2023-12-31T04:41:18.000Z

took a while to figure out: it works both normally and through gamescope. really tired now. will post how to do this later :)

short update:
the thing I am doing is not a good solution and should be avoided.
there is a proper (not recommended) way to do it though. currently seeking help with that :)

edit:
this may take a long while...

Answer 26 · 2023-12-31T12:56:06.000Z

this may take a long while...

I'm not in a hurry, it was great to have a proof of concept so quick already :)
Better take the time and do things properly :)

Answer 27 · 2024-02-24T19:41:55.000Z

took a while to figure out: it works both normally and through gamescope. really tired now. will post how to do this later :)

short update: the thing I am doing is not a good solution and should be avoided. there is a proper (not recommended) way to do it though. currently seeking help with that :)

edit: this may take a long while...

follow-up?

Answer 28 · 2024-03-05T15:22:15.000Z

+1 for an update.
I tried to use it for Enshrouded to fix its awful bloom / brightness issues but as far as I can tell it is using Vulkan exclusively. Unfortunately Goverlay currently also has VKBasalt support removed in their UI overhaul and I don't think there's any other UI helper for it, let alone an in-game UI, which makes it hard to work with too.

Answer 29 · 2024-03-05T16:00:21.000Z

Current working on other projects which should be done this week.
Then I can get to this :)

Answer 30 · 2024-04-22T17:17:52.000Z

Current working on other projects which should be done this week. Then I can get to this :)

any update on this?

Answer 31 · 2024-05-12T12:21:58.000Z

Any news? I am looking forward to jumping back to Linux for all my Vulkan games:)

Answer 32 · 2024-05-14T09:00:33.000Z

I am switching between 5 projects to not get burned out 🙃
I want go finish some other stuff first before getting to finalize this.

I have it working in Vulkan but only in Proton 8. Not sure what the issue is in other Proton versions. Also the DirectX part of the script is not working atm (I rewrote it already but haven't tested it yet).
I also haven't looked into Wine at all yet.

Answer 33 · 2024-05-24T09:34:27.000Z

you can find the current state here:
https://github.com/EndlesslyFlowering/reshade-steam-proton

currently I am only focusing on the reshade-steam-proton script:

you need to be on Proton 8.0 or later! I have only tested official Proton versions from Valve (Proton 8.0-x and Proton 9.0-x) and not GE variants. I have had reports of GE not working, so take that as is
you need protontricks either via flatpak or normally installed (the script checks if it's available)
~~the script is also incomplete because I added some features and did not implement them yet for DirectX games (I actually implemented them already but did not yet test them...)~~

Answer 34 · 2024-05-24T12:57:54.000Z

DirectX games should work again. I did some testing and everything works.

Answer 35 · 2024-06-17T02:17:46.000Z

Were you able to get Reshade working for linux/proton games on steam that are vulkan games? I'm trying to do it on Doom Eternal but I don't know where to start

Answer 36 · 2024-06-27T09:56:29.000Z

Were you able to get Reshade working for linux/proton games on steam that are vulkan games? I'm trying to do it on Doom Eternal but I don't know where to start

Yes Proton only for now.
Script asks you if it is a Vulkan game.