ldconfig warns about truncated libraries
darealshinji opened this issue · 26 comments
ldconfig puts a warning on files that were modified with patchelf:
djcj patchelf-master $ sudo src/patchelf --set-rpath /usr/lib /usr/lib/x86_64-linux-gnu/libx264.so.142
djcj patchelf-master $ LANG=C sudo ldconfig
/sbin/ldconfig.real: file /usr/lib/x86_64-linux-gnu/libx264.so.142 is truncated
djcj patchelf-master $
The sections .dynsym and .dynstr are moved to the end of the file. Originally .shstrtab was the last section. That might cause the trouble.
I am getting the same error when using the --set-soname option on 63296c4.
I'm seeing this error as well. Do you have a fix or workaround?
For me with --set-soname, I am pretty sure replaceSection() is broken. Since I am always shortening the name, I have a custom patch that detects I am calling it and does the name replacement directly in memory by zeroing out the unused bytes at the end. The original code always assumed the new name was longer and called replaceSection().
I use --set-interpreter and --set-rpath and they usually get longer for me. Thanks for the tip though. I could maybe arrange it for them to get shorter rather than longer. Probably not always though.
@AaronDMarasco-VSI Can you provide said custom patch?
@darealshinji It doesn't fix replaceSection(); instead it bypasses it in a very specific way that would be pretty useless for most people. I compile the .so with the suffix "_s" but then later rename it to without so need to change that name explicitly.
Here's the patch, but like I said, extremely specific and unlikely to be helpful for public consumption.
It looks like ldconfig expects .dynstr to be in the first LOAD segment (elf/readelf.c in glibc), although I don't know why.
It looks like the code in readelflib.c simply searches for the very first section with type STRTAB, and uses that.
For me, patchelf seems to change the order of the sections, so e.g. before patchelf is applied, a shared library I have has the following section headers:
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000 000000 00 0 0 0
[ 1] .note.gnu.build-id NOTE 00000000000001c8 0001c8 000024 00 A 0 0 4
[ 2] .gnu.hash GNU_HASH 00000000000001f0 0001f0 0001e8 00 A 3 0 8
[ 3] .dynsym DYNSYM 00000000000003d8 0003d8 001800 18 A 4 2 8
[ 4] .dynstr STRTAB 0000000000001bd8 001bd8 0011c1 00 A 0 0 1
[ 5] .gnu.version VERSYM 0000000000002d9a 002d9a 000200 02 A 3 0 2
[ 6] .gnu.version_d VERDEF 0000000000002fa0 002fa0 0000dc 00 A 4 7 8
[ 7] .gnu.version_r VERNEED 0000000000003080 003080 000140 00 A 4 7 8
[ 8] .rela.dyn RELA 00000000000031c0 0031c0 030b10 18 A 3 0 8
[ 9] .rela.plt RELA 0000000000033cd0 033cd0 001008 18 A 3 11 8
[10] .init PROGBITS 0000000000034cd8 034cd8 00000e 00 AX 0 0 4
[11] .plt PROGBITS 0000000000034cf0 034cf0 000ac0 10 AX 0 0 16
[12] .text PROGBITS 00000000000357b0 0357b0 7460b0 00 AX 0 0 16
[13] .fini PROGBITS 000000000077b860 77b860 000009 00 AX 0 0 4
[14] .rodata PROGBITS 000000000077b880 77b880 0c8da1 00 A 0 0 64
[15] .eh_frame_hdr PROGBITS 0000000000844624 844624 0132c4 00 A 0 0 4
[16] .eh_frame PROGBITS 00000000008578e8 8578e8 088b2c 00 A 0 0 8
[17] .gcc_except_table PROGBITS 00000000008e0414 8e0414 00749c 00 A 0 0 4
[18] .init_array INIT_ARRAY 0000000000ae7ec8 8e7ec8 0000c0 00 WA 0 0 8
[19] .fini_array FINI_ARRAY 0000000000ae7f88 8e7f88 000008 00 WA 0 0 8
[20] .jcr PROGBITS 0000000000ae7f90 8e7f90 000008 00 WA 0 0 8
[21] .data.rel.ro PROGBITS 0000000000ae7fa0 8e7fa0 010538 00 WA 0 0 32
[22] .dynamic DYNAMIC 0000000000af84d8 8f84d8 000270 10 WA 4 0 8
[23] .got PROGBITS 0000000000af8748 8f8748 0008a0 08 WA 0 0 8
[24] .data PROGBITS 0000000000af9000 8f9000 012360 00 WA 0 0 32
[25] .bss NOBITS 0000000000b0b360 90b360 00fd40 00 WA 0 0 32
[26] .gnu_debuglink PROGBITS 0000000000000000 90b360 000018 00 0 0 1
[27] .shstrtab STRTAB 0000000000000000 90b378 00010d 00 0 0 1
so the .dynstr section (no. 4) is the first STRTAB, and the .shstrtab section (no. 27) is the second (and basically ignored by ldconfig).
But after setting a DT_RUNPATH entry with patchelf --set-rpath, the headers become:
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000 000000 00 0 0 0
[ 1] .dynsym DYNSYM 00000000000003d8 0003d8 001800 18 A 25 2 8
[ 2] .gnu.version VERSYM 0000000000002d9a 002d9a 000200 02 A 1 0 2
[ 3] .gnu.version_d VERDEF 0000000000002fa0 002fa0 0000dc 00 A 25 7 8
[ 4] .gnu.version_r VERNEED 0000000000003080 003080 000140 00 A 25 7 8
[ 5] .rela.dyn RELA 00000000000031c0 0031c0 030b10 18 A 1 0 8
[ 6] .rela.plt RELA 0000000000033cd0 033cd0 001008 18 A 1 8 8
[ 7] .init PROGBITS 0000000000034cd8 034cd8 00000e 00 AX 0 0 4
[ 8] .plt PROGBITS 0000000000034cf0 034cf0 000ac0 10 AX 0 0 16
[ 9] .text PROGBITS 00000000000357b0 0357b0 7460b0 00 AX 0 0 16
[10] .fini PROGBITS 000000000077b860 77b860 000009 00 AX 0 0 4
[11] .rodata PROGBITS 000000000077b880 77b880 0c8da1 00 A 0 0 64
[12] .eh_frame_hdr PROGBITS 0000000000844624 844624 0132c4 00 A 0 0 4
[13] .eh_frame PROGBITS 00000000008578e8 8578e8 088b2c 00 A 0 0 8
[14] .gcc_except_table PROGBITS 00000000008e0414 8e0414 00749c 00 A 0 0 4
[15] .init_array INIT_ARRAY 0000000000ae7ec8 8e7ec8 0000c0 00 WA 0 0 8
[16] .fini_array FINI_ARRAY 0000000000ae7f88 8e7f88 000008 00 WA 0 0 8
[17] .jcr PROGBITS 0000000000ae7f90 8e7f90 000008 00 WA 0 0 8
[18] .data.rel.ro PROGBITS 0000000000ae7fa0 8e7fa0 010538 00 WA 0 0 32
[19] .got PROGBITS 0000000000af8748 8f8748 0008a0 08 WA 0 0 8
[20] .data PROGBITS 0000000000af9000 8f9000 012360 00 WA 0 0 32
[21] .bss NOBITS 0000000000b0b360 90b360 00fd40 00 WA 0 0 32
[22] .gnu_debuglink PROGBITS 0000000000000000 90b360 000018 00 0 0 1
[23] .shstrtab STRTAB 0000000000000000 90b378 00010d 00 0 0 1
[24] .dynamic DYNAMIC 0000000000b1c000 90c000 000280 10 WA 25 0 8
[25] .dynstr STRTAB 0000000000b1c280 90c280 0011f9 00 A 0 0 8
[26] .gnu.hash GNU_HASH 0000000000b1d480 90d480 0001e8 00 A 1 0 8
[27] .note.gnu.build-id NOTE 0000000000b1d668 90d668 000024 00 A 0 0 8
so now .shstrtab has become number 23, and moved before .dynstr, while patchelf has inserted the DT_RUNPATH entry into that table.
Apparently the readelf utility and the dynamic linker itself do not care about this different ordering, but ldconfig now uses the wrong string table, and it will always think the DT_RUNPATH string falls outside the range for that table.
I may be wrong but I think I read somewhere that ELF specification doesn't require where the sections are written, so I guess readelf's and ld.so's behavior are right while ldconfig's behavior is a bug.
Oh certainly, I agree, ldconfig should search specifically for .dynstr, or in fact maybe consider all string tables together. However, even if you would submit a glibc fix, and got it accepted, lots of distros out there come with a (very) old version, showing this error. (I think it also stops processing that particular library, which is annoying.)
Good sleuthing, Dimitry! A patch upstream to Glibc is definitely warranted. In the mean time, a workaround for patchelf that ensures that .dynstr is the first STRTAB section would be useful. Thanks for sorting out the root cause!
I don't think the issue is caused by the wrong order of .dynstr and .shstrtab. I forced patchelf to overwrite the .shstrtab section too, with the following hack:
--- patchelf_orig.cc 2018-04-21 15:39:40.057741000 +1000
+++ patchelf.cc 2018-04-21 16:16:20.097747164 +1000
@@ -618,6 +618,12 @@
s.resize(size);
replacedSections[sectionName] = s;
+ if (sectionName==".dynstr" && replacedSections.find(".shstrtab")==replacedSections.end()) {
+ Elf_Shdr & shdr = findSection(".shstrtab");
+ s = std::string((char *) contents + rdi(shdr.sh_offset), rdi(shdr.sh_size));
+ replacedSections[".shstrtab"] = s;
+ }
+
return replacedSections[sectionName];
}The sections became reversed again but it didn't make ldconfig happy. Meanwhile, they had been written to a new separate segment LOAD at the end (which can be seen by readelf -l), so I suspect the @Narthorn 's version is more plausible.
is there any solution to fix this or work around that problem?
could it be that some string is missing?
if I try to read the rpath via "chrpath" utility, Ill get
RUNPATH string offset not contained in string table
This is probably better described as an "ldconfig error" rather than an "ldconfig warning" as in the title. It prints to standard error and the dynamic loader can't find the libraries afterwards.
Here's a small patch in case anyone else needs ldconfig to work:
NixOS/nixpkgs@d44a939#diff-64b83ddc8edce55f9aff3a4e639ce3bd
Basically as @Narthorn said, the .dynstr vaddr is resolved against the first LOAD segment, which doesn't work on any libs where patchelf called rewriteSections, because that moves .dynstr to the last LOAD segment.
Additionally I don't think it's possible to fix this in patchelf itself in the current implementation:
- If soname gets longer, or rpath is added, dynstr cannot fit at it's old location.
- LOAD segments have to be referenced in order of ascending vaddr in the header (I checked the specs)
This seems like it's actually a bug in ldconfig. process_elf_file is assuming that the entire file will be mapped contiguously, which of course isn't required by the spec. It'd probably be worth filing against glibc.
It is, here's a bug report filed in December: https://sourceware.org/bugzilla/show_bug.cgi?id=23964. But it's probably not a priority, or at least there hasn't been any activity.
Not sure if this is the right place to ask, but does anybody know if this issue is being fixed in ldconfig upstream?
As I understand it, this is an issue already noted 10 years ago, e.g. here https://nix-dev.science.uu.narkive.com/q6Ww5fyO/ldconfig-problem-with-patchelf-and-64-bit-libs
This problem exists in glibc 2.30 but does not in glibc 2.31, so this has probably been fixed in glibc upstream.
[root@rosa-2019 lib64]# ldconfig -V | head -n 1
ldconfig (GNU libc) 2.31
[root@rosa-2019 lib64]# ldconfig -l libstdc++-gcc10.so.6.0.28
[root@rosa-2019 lib64]#
user@pay2:/tmp$ LC_ALL=C ldconfig -V | head -n 1
ldconfig (Ubuntu GLIBC 2.30-0ubuntu2.1) 2.30
user@pay2:/tmp$ LC_ALL=C ldconfig -l libstdc++-gcc10.so.6.0.28
/sbin/ldconfig.real: file libstdc++-gcc10.so.6.0.28 is truncated
/sbin/ldconfig.real: No link created since soname could not be found for libstdc++-gcc10.so.6.0.28
Thanks, give it 3 more years and Ubuntu 18.04 is end of life, then we don't have to worry about this issue anymore 😅
then we don't have to worry
<cries in CentOS 7>
Given this is fixed in glibc I'm closing this issue.
This issue has been fixed in glibc 2.31. #44 (comment)
glibc 2.31 is included in Ubuntu 20.04 and Fedora 32, but nothing older, in particular no stable release of either Debian or CentOS.
https://github.com/Linuxbrew/brew/wiki/Bottles#glibc-version
(for my and other's information)
Also see ldconfig and file xxx is truncated in Ubuntu's Launchpad.