chros73/rtorrent-ps-ch

Fix rpath linking on newer distros by using relative rpath linking

chros73 opened this issue · 7 comments

chros73 commented

It's seems that all of the necessary xmlrpc's so file can be found at the compile time, but some of them can't be found at runtime without change LD_LIBRARY_PATH.(build with ./build.sh all, of course firstly comment out line 133)
the command of (ldd rtorrent)'s output:

libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fd3df4a0000)
        libcurl.so.4 => /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libcurl.so.4 (0x00007fd3df235000)
        libtorrent.so.19 => /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libtorrent.so.19 (0x00007fd3def02000)
        libxmlrpc_server.so.3 => /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libxmlrpc_server.so.3 (0x00007fd3decfb000)
        libxmlrpc.so.3 => /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libxmlrpc.so.3 (0x00007fd3deadd000)
        libxmlrpc_util.so.3 => /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libxmlrpc_util.so.3 (0x00007fd3de8d7000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd3de6b9000)
        /lib64/ld-linux-x86-64.so.2 (0x0000564f45cd9000)
        libxmlrpc_xmlparse.so.3 => not found

strange thing is libxmlrpc_xmlparse.so.3 not found, but
libxmlrpc_server.so.3,
libxmlrpc.so.3,
libxmlrpc_util.so.3
can be found. In fact all of them are in the same directory.

Only ubuntu 17.07 has this problem:
ubuntu version: ubuntu-server 4.10.0-22-generic #24-Ubuntu SMP Mon May 22 17:43:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

pyroscope commented

Not strange, the reason is in my start script changesets.

chros73 commented

Thanks, @pyroscope for the reminder, I remember your change for about 3 months ago. I'll just want to play with it, maybe there's an easier solution for it for now.

chros73 commented

@pyroscope : actually we can, without that hack:

A. theory
https://stackoverflow.com/questions/7967848/use-rpath-but-not-runpath

  • if RUNPATH AND RPATH are both set then RUNPATH applies (although newer systems only set RUNPATH)

Why libxmlrpc_xmlparse.so.3 is not found when the RUNPATH is set?

  • from this post:
    "the direct deps are in the ELF's NEEDED-field and objdump -p return these. So it turns out, libglobalshortcuts.so is not a direct dependency to albert, while libalbertcore.so is and also gets found on Zesty"

That's exactly our case: libxmlrpc_xmlparse.so.3 is not direct dependency of rtorrent hence looking it up fails when ONLY RUNPATH is set. (LD_DEBUG=libs ldd ./rtorrent)

So there are 2 possible options for resolving this:

  • add that lib to rtorrent as direct dep (I don't know how?)
  • force using RPATH instead of RUNPATH

But, from one of the comments (of the above linked stackoverflow link): "it may make sense for the application vendors, which are generally concerned only with their own application and not with the big picture. But it does not make sense from the point of view of system vendors and integrators and it's them who deprecated DT_RPATH"

So, in our case it's a complete nonsense to use RUNPATH: since we ship all the main required libraries and we do this only for 1 reason, which is the user should use those libraries!

In short, what we want to do is apply RPATH only:

  • we have to set RPATH
  • we have to get rid of the RUNPATH setting

B. Diagnostic commands

objdump -p ./rtorrent
chrpath -l ./rtorrent
ldd ./rtorrent

C. Dirty hack for now
To see whether the theory works indeed, we can modify the rtorrent binary (we only have to mod this file), like this (unfortunately we need 2 utils for this, more info):

  • delete the set RUNPATH entry from the binary
  • add the proper RPATH entry to the binary
chrpath -d rtorrent
patchelf --force-rpath --set-rpath /home/chros73/lib/rtorrent1/lib rtorrent

If I can't find out how to force only RPATH on these systems in the proper way then I'll implement this hack (only for these systems).

D. Relative rpath linking
Probably you remember when I asked how to achieve this.
I just came across the solution (since I knew now what I'm looking for):

  • modifying the 2nd command in the previous hack solves this problem :)
    • from the post: "-L adds a path only to the build-time library search path list. (Note: rpath is irrelevant at build-time, -L is irrelevant at runtime).
      -Wl,-rpath,'' embeds into the generated library only as a runtime library search path"
patchelf --force-rpath --set-rpath '$ORIGIN/../lib' rtorrent

With only this modification, the whole directory can be moved anywhere, binary can be started from anywhere.

Edit: I've extended D. a bit.

chros73 commented

@pyroscope , one more thing, having the above knowledge (and I've also tested on Ubuntu 17.04):

  • you don't need this line
    -re 's:^NEED_WL_RPATH=.+$:NEED_WL_RPATH="yes":' \
    • it doens't do anything, since libxmlrpc_xmlparse.so.3 doesn't have any RUNPATH entries by default
    • it's entirely the fault of rtorrent binary
  • setting only LD_LIBRARY_PATH solves your issue
chros73 commented

Woow! That was unexpected!

A.
Using only relative rpath linking solves all our problems, even with "RUNPATH" as well!
All the posts, articles failed to mention this!

The good thing about this that we don't have to modify any of the distro specific settings.

Eg. on Ubuntu 14.04:
without relative rpath:

$ objdump -p ~/lib/rtorrent-0.9.7-1.5.3/bin/rtorrent | grep PATH
  RPATH                /home/chros/lib/rtorrent-0.9.7-1.5.3/lib

~/bin/rtorrent-0.9.7 -> ~/lib/rtorrent-0.9.7-1.5.3/bin/rtorrent-extended
        libcurl.so.4 => /home/chros/lib/rtorrent-0.9.7-1.5.3/lib/libcurl.so.4 (0x00007f2d0ecc4000)
        libtorrent.so.19 => /home/chros/lib/rtorrent-0.9.7-1.5.3/lib/libtorrent.so.19 (0x00007f2d0e99c000)
        libxmlrpc_server.so.3 => /home/chros/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc_server.so.3 (0x00007f2d0e795000)
        libxmlrpc.so.3 => /home/chros/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc.so.3 (0x00007f2d0e57d000)
        libxmlrpc_util.so.3 => /home/chros/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc_util.so.3 (0x00007f2d0e377000)
        libcares.so.2 => /home/chros/lib/rtorrent-0.9.7-1.5.3/lib/libcares.so.2 (0x00007f2d0d15d000)
        libxmlrpc_xmlparse.so.3 => /home/chros/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc_xmlparse.so.3 (0x00007f2d0c6fa000)
        libxmlrpc_xmltok.so.3 => /home/chros/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc_xmltok.so.3 (0x00007f2d0c4de000)

with relative rpath:

$ objdump -p ~/lib/rtorrent-0.9.7-1.5.3/bin/rtorrent | grep PATH
  RPATH                $ORIGIN/../lib:/home/chros/lib/rtorrent-0.9.7-1.5.3/lib

~/bin/rtorrent-0.9.7 -> ~/lib/rtorrent-0.9.7-1.5.3/bin/rtorrent-extended
        libcurl.so.4 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libcurl.so.4 (0x00007f3dcb4a8000)
        libtorrent.so.19 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libtorrent.so.19 (0x00007f3dcb180000)
        libxmlrpc_server.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc_server.so.3 (0x00007f3dcaf79000)
        libxmlrpc.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc.so.3 (0x00007f3dcad61000)
        libxmlrpc_util.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc_util.so.3 (0x00007f3dcab5b000)
        libcares.so.2 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libcares.so.2 (0x00007f3dc993f000)
        libxmlrpc_xmlparse.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/../lib/libxmlrpc_xmlparse.so.3 (0x00007f3dc8edd000)
        libxmlrpc_xmltok.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/../lib/../lib/libxmlrpc_xmltok.so.3 (0x00007f3dc8cc0000)

Eg. on Ubuntu 17.04:
with relative rpath:

$ objdump -p ~/lib/rtorrent-0.9.7-1.5.3/bin/rtorrent | grep PATH
  RUNPATH              $ORIGIN/../lib:/home/chros/lib/rtorrent-0.9.7-1.5.3/lib

~/bin/rtorrent-0.9.7 -> ~/lib/rtorrent-0.9.7-1.5.3/bin/rtorrent-extended
	libcurl.so.4 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libcurl.so.4 (0x00007fde98ac5000)
        libtorrent.so.19 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libtorrent.so.19 (0x00007fde98791000)
        libxmlrpc_server.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc_server.so.3 (0x00007fde9858a000)
        libxmlrpc.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc.so.3 (0x00007fde9836c000)
        libxmlrpc_util.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libxmlrpc_util.so.3 (0x00007fde98166000)
        libcares.so.2 => ~/lib/rtorrent-0.9.7-1.5.3/lib/libcares.so.2 (0x00007fde96ec0000)
        libxmlrpc_xmlparse.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/../lib/libxmlrpc_xmlparse.so.3 (0x00007fde963e5000)
        libxmlrpc_xmltok.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/../lib/../lib/libxmlrpc_xmltok.so.3 (0x00007fde961c6000)

So, the interesting part are these lines:

        libxmlrpc_xmlparse.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/../lib/libxmlrpc_xmlparse.so.3 (0x00007fde963e5000)
        libxmlrpc_xmltok.so.3 => ~/lib/rtorrent-0.9.7-1.5.3/lib/../lib/../lib/libxmlrpc_xmltok.so.3 (0x00007fde961c6000)

It seems, when relative path is used during rpath linking then (based on the level of dependency) the linker can jump up/down again-and-again, but not when full path is set.

B.
Relative rpath linking has the following good side effect, main directory can be:

  • renamed
  • moved
  • copied !!!
    • it means if your compile as a user was good, then you don't have to recompile it with install parameter but just copy the whole directory into /opt dir!

C.
Tested on:

  • Ubuntu 14.04
  • Ubuntu 17.04
  • Debian 9
chros73 commented

I want to figure it out how we can remove the extra unneeded absolute path from the property:

  • libtool auto-puts it (-L flag in LDFLAGS)
chros73 commented

It seems, when relative path is used during rpath linking then (based on the level of dependency) the linker can jump up/down again-and-again, but not when full path is set.

It turned out upon further investigation, that not the relative rpath linking what made it work but the extra -Wl,-rpath,* flag in LDFLAGS.

Actually specifying the full rpath once more in LDFLAGS also works:
export LDFLAGS="-L$INST_DIR/lib -Wl,-rpath,$INST_DIR/lib${LDFLAGS:+ }${LDFLAGS}"

So, in summary, why it works:

  • RPATH: instructs the binary to look for all the libraries in the given paths (not just direct dependencies)
  • RUNPATH: instructs the binary to look for only directly dependent libraries

When -Wl,-rpath,* flag is also added into LDFLAGS then all the compiled binaries (executables, libraries) will have the RPATH/RUNPATH entry, not just those that was supposed to have it (with their configure script), e.g. libcares.so, all the libxml*.so, etc:

  • this what happens when only RUNPATH is available in the binary
    • the binary finds its direct descendant
    • then the given direct descendant will find for the binary it's own direct descendant
    • and so on

That's why libxmlrpc_xmltok.so.3 appeared in the filtered ldd list.

  • 👍1