felixdoerre/primus_vk

pvkrun unable to launch Terraria 1.4 from steam (pvkrun %command%)

x-qq opened this issue · 13 comments

x-qq commented

I am trying to launch Terraria 1.4 using pvkrun from inside steam using a custom launch command but the process immediately terminates.

In ~/.steam/error.log I see this:

primus: fatal: failed to load any of the libraries: /usr/lib/x86_64-linux-gnu/nvidia/libGL.so.1:/usr/lib/i386-linux-gnu/nvidia/libGL.so.1:/usr/lib/nvidia/libGL.so.1
/usr/lib/x86_64-linux-gnu/nvidia/libGL.so.1: cannot open shared object file: No such file or directory
/usr/lib/i386-linux-gnu/nvidia/libGL.so.1: cannot open shared object file: No such file or directory
/usr/lib/nvidia/libGL.so.1: cannot open shared object file: No such file or directory
Game removed: AppID 105600 "", ProcID 52291 
Uploaded AppInterfaceStats to Steam
Exiting app 105600

Launching the game directly as pvkrun ./Terraria.bin.x86_64 works.
Launching steam as pvkrun steam and then launching the game from it (without a custom launch command) works too.

So it seems like there is some vulkan-related conflict.

primus-vk-nvidia 1.4-1 on Debian sid amd64

felixdoerre commented

Can you give a bit more information about what nvidia driver is installed?
Maybe with the output of these commands:

$ dpkg --get-selections | grep nvidia
$ ls -als /usr/lib/*/libGLX_nvidia.*

The error itself does not contain any vulkan-related information. To me this looks as a plain primus/glvnd-driver problem. I currently don't see any indication that this is pvkrun-related.

Additionally, are you sure that the game uses vulkan and does not just use OpenGL?

x-qq commented

I am using nvidia-driver 440.82-2 on kernel 5.6.14-1. The GPU is GTX 1050Ti Mobile (10de:1c8c).

% dpkg --get-selections | grep nvidia
glx-alternative-nvidia				install
libegl-nvidia0:amd64				install
libegl-nvidia0:i386				install
libgl1-nvidia-glvnd-glx:amd64			install
libgl1-nvidia-glvnd-glx:i386			install
libgles-nvidia1:amd64				install
libgles-nvidia1:i386				install
libgles-nvidia2:amd64				install
libgles-nvidia2:i386				install
libglx-nvidia0:amd64				install
libglx-nvidia0:i386				install
libnvidia-cfg1:amd64				install
libnvidia-eglcore:amd64				install
libnvidia-eglcore:i386				install
libnvidia-glcore:amd64				install
libnvidia-glcore:i386				install
libnvidia-glvkspirv:amd64			install
libnvidia-glvkspirv:i386			install
libnvidia-ml1:amd64				install
nvidia-alternative				install
nvidia-driver					install
nvidia-driver-bin				install
nvidia-driver-libs:amd64			install
nvidia-driver-libs:i386				install
nvidia-driver-libs-i386:i386			install
nvidia-egl-common				install
nvidia-egl-icd:amd64				install
nvidia-egl-icd:i386				install
nvidia-installer-cleanup			install
nvidia-kernel-4.2.0-1-amd64			deinstall
nvidia-kernel-4.3.0-1-amd64			deinstall
nvidia-kernel-common				install
nvidia-kernel-dkms				install
nvidia-kernel-support				install
nvidia-legacy-check				install
nvidia-modprobe					install
nvidia-persistenced				install
nvidia-settings					install
nvidia-support					install
nvidia-vdpau-driver:amd64			install
primus-vk-nvidia				install
primus-vk-nvidia-i386:i386			install
xserver-xorg-video-nvidia			install
% 

% ls -als /usr/lib/*/libGLX_nvidia.*
0 lrwxrwxrwx 1 root root 59 May 28 18:52 /usr/lib/i386-linux-gnu/libGLX_nvidia.so.0 -> /etc/alternatives/nvidia--libGLX_nvidia.so.0-i386-linux-gnu
0 lrwxrwxrwx 1 root root 61 May 28 18:52 /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.0 -> /etc/alternatives/nvidia--libGLX_nvidia.so.0-x86_64-linux-gnu
%

I don't know for sure if the error is vulkan-related.

felixdoerre commented

Ok, this looks for sure like the glvnd-driver is correctly installed. I presume that this is a problem in the primus/bumblebee combination not detecting the glvnd-driver correctly and the hint from outside from prkrun does not get through to the game. I imagine that the next version of the primus+bumblebee packages that are currently worked on can fix this problem.

Are you using pvkrun %command% as custom launch command? The shown error would totally make sense for primusrun %command% or optirun %command%. It looks like the environment variable PRIMUS_libGLa that pvkrun 1.4-1 on debian sets as workaround for the glvnd-driver is missing.

Another test that might bring inside, would be to use the launcher that steam itself uses (i.e. pvkrun ./Terraria). Does that work as well or show the problem from the steam output.

x-qq commented

Yes, I am using pvkrun %command% in steam.

Launching Terraria directly like pvkrun ./Terraria.bin.x86_64 works (I mentioned this in the first post).

felixdoerre commented

I suggested you try to use the Terraria-shellscript which is a wrapper around the binaries, but I tested that locally, and it seems to work for me (but the error message I get when running from steam is different)

I have the theory that this problem is caused by exactly this Terraria-shellscript-wrapper. You can try this by setting the launch command from steam to: pvkrun ./Terraria.bin.x86_64 #%command%. When I try this, this works as a workaround. Can you verify that as well?

x-qq commented

Yes, I can verify that launch command pvkrun ./Terraria.bin.x86_64 #%command% works.

Additionally, using the shellscript directly from outside steam as pvkrun ./Terraria works too.

felixdoerre commented

Hi, I am pretty sure now, that I understand what goes wrong here and am currently trying to understand how to address this best, with some debian developers as this is a debian-specific problem. I am also fairly certain that this is not a primus-vk problem, but rather a primus problem, but I am still trying to find a solution and get this resolved.

If you are interested in the details of what happens, please leave a note and I can write a complete explanation. Otherwise I will update this issue when I have news on when/how this will be fixed in debian.

x-qq commented

Hi, I would be interested in a description of what causes the problem, I am using Debian after all.

felixdoerre commented

Ok, What is involved in the problem:

  1. We LD_PRELOAD=: gameoverlayrenderer.so. Steam always does this for the Steam overlay. This is missing when the game is launched from outside of Steam.
  2. In order to run a game with primus, we need to set LD_LIBRARY_PATH in order to load the libGL.so.1 from primus instead of the normal libGL.so.1.
  3. The game is launched by a /bin/bash wrapper.

What happens:

  1. bash is started.
  2. gameoverlayrenderer.so is preloaded
  3. as gameoverlayrenderer.so is directly dependent on libGL.so.1, we load the primus libGL.so.1 as a dependency
  4. the primus libGL.so.1 sets the environment variable __GLVND_DISALLOW_PATCHING, by calling putenv: (https://salsa.debian.org/nvidia-team/primus/-/blob/23a9620ef33d5a623d55731c6c8ca588fdb65a28/debian/patches/glvnd.patch#L10). This is required to prevent glvnd from courageously binary-patching OpenGL-Api functions inside the RAM when the OpenGL context is changed (https://github.com/NVIDIA/libglvnd/blob/bed48a107cc0d3a810ff642f5d013101ae3f4c63/src/GLdispatch/GLdispatch.c#L340). This is not good for primus, as it wants to access two different contexts simultaneously.
  5. However, putenv is Hijacked by bash: https://salsa.debian.org/debian/bash/-/blob/6a0146056618a32be35ba89e1b092eda2f2fav.c#L101 this is a mechanism (or Hack?) by bash to quickly export its own environment to library functions. This own environment from is not initialized yet, as the main()-function form bash has not been called yet. We are still initializing libraries at this point. So putenv, does not really modify the environment, but only stores this environment variable in the bash shell_variables-structure.
  6. Primus calls XOpenDisplay(NULL) (which is forwarded by gameoverlayrenderer.so to libX11.so), which calls getenv("DISPLAY"). getenv is also hijacked by bash. However the external variables are still not imported. Bash has a mechanism to forward such calls to the "real" environment (https://salsa.debian.org/debian/bash/-/blob/6a0146056618a32be35ba89e1b092eda2f2fa749/lib/sh/getenv.c#L74). This mechanism only activates when the shell_variables-structure is absent. The previous call to putenv however already initialized this structure. So we don't find a DISPLAY environment variable, and libX11.so returns an error.

What is wrong here? What could be done to resolve this problem?

There are several alternatives:

  1. the mechanism in bash is just wrong as it breaks the environment when it is used before bash's main() is started (Which would have happend after all the steps I described above). Interestingly it is only broken if one calls putenv (or setenv) before calling getenv, so not many applications go into this trap. So as a resolution putenv (and setenv) from bash should be changed to directly forward to libc's putenv (or setenv), if the bash variables have not been initialized yet.
  2. Do not set any environment variable from within primus. So we would remove the debian-patch and set __GLVND_DISALLOW_PATCHING outside, from primusrun or optirun.
  3. Do not use putenv/setenv but instead directly modify the environ libc-internal global variable. This would really be a hack.
  4. Use dlopen/dlsym to get the location of the real setenv function and call this from primus instead.

Note: Interestingly this kind-of also might affect primus-vk, as it also sets an environment variable here: https://github.com/felixdoerre/primus_vk/blob/master/nv_vulkan_wrapper.cpp#L88
However this is not a problem (yet) as gameoverlayrenderer.so does not load libvulkan and thereby libnv_vulkan_wrapper (at least not from within the early initializers).

felixdoerre commented

Hi, there was just uploaded a primus version that should solve/work around this issue into debian sid: 0~20150328-11 You should receive it in the next hours via normal system updates. Could you re-test and report back when you have received the update?

Just for reference: the bug against the debian's bash package is opened here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=962566 Hopefully the maintainer will submit this bug upstream and the issue gets solved, we will see.

felixdoerre commented

No, that's correct: primus-libs-ia32 is an outdated mechanism that did never contain any real files but was just use to recommend the 32-bit version of primus-libs. Now this recommendation is directly included in the primus-libs-package, which directly recommends its 32-bit version: https://salsa.debian.org/nvidia-team/primus/-/blob/412b48fa44080669e38d9e09f9601c48d9f421da/debian/control#L46

So summing it up: the removal of primus-libs-ia32 is a cleanup that is expected.

x-qq commented

Thanks, the fix works. With updated primus-libs package I am able to use launch command pvkrun %command% now.

felixdoerre commented

Great, so I assume this issue is resolved now and close it. If you have any other problems with primus_vk feel free to open a new issue or re-open this issue if appropriate.