loadable library and perl binaries are mismatched
dkwo opened this issue · 21 comments
on Void Linux (x86_64-musl), there is the package texlive2023-bin. it installs texlive distribution in /opt with its package manager tlmgr. then one can install biber by tlmgr install biber. when i use it, it produces
$ biber --help
IO.c: loadable library and perl binaries are mismatched (got first handshake key 0xe580080, needed 0xeb80080)
do you know what the issue could be?
among other things, the texlive2023-bin does
cd opt/texlive2023-installer
./install-tl -profile void.profile -force-platform x86_64-linuxmusl
where
# texlive.profile written for voidlinux
TEXDIR ../texlive/2023
TEXMFCONFIG ~/.texlive2023/texmf-config
TEXMFHOME ~/texmf
TEXMFLOCAL ../texlive/texmf-local
TEXMFSYSCONFIG ../texlive/2023/texmf-config
TEXMFSYSVAR ../texlive/2023/texmf-var
TEXMFVAR ~/.texlive2023/texmf-var
Bringing in @krumeich as he knows about those binaries. Are you sure you are using the "binary" version of biber? What does which biber say? There is a MUSL binary but I wonder if TL detected the right OS?
$ which biber
/opt/texlive/2023/bin/x86_64-linuxmusl/biber
I've just tried running the biber binary (just the way that it falls out of my build chain) in a voidlinux docker image, but no success: The shell wouldn't even start the program.
Under Alpine, I tried to see where IO.c was opened during startup with strace -e open,stat,lstat:
strace -e open,stat,lstat biber 2>&1 | grep IO.c
However, this didn't print any results.
Do you have any environment variables that point to different perl installations? biber for x86_64-linuxmusl is built against perl 5.36. Could that be a version conflict on your distro?
there's perl 5.36.0 installed as a package system-wide, it provides perl-IO-1.50_1.
in printenv i see PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/texlive/2023/bin/x86_64-linuxmusl:/home/dkwo/bin
does biber from tlmgr use its own perl or system perl?
the perl-io version is obtained from a script similar to arch linux, at build time.
do you build biber with the same version of perl-io or newer one?
Perl shouldn't have to be installed at all for the TL biber, it should be entirely self-contained. If it is impacted by an installed perl, it means that the packaging didn't pack all dependencies.
For this, I'd need a helping hand because I don't know enough about Perl libraries. My build script for PAR::Packer includes the following modules:
--module=deprecate \
--module=Biber::Input::file::bibtex \
--module=Biber::Input::file::biblatexml \
--module=Biber::Output::dot \
--module=Biber::Output::bbl \
--module=Biber::Output::bblxml \
--module=Biber::Output::bibtex \
--module=Biber::Output::biblatexml \
--module=Pod::Simple::TranscodeSmart \
--module=Pod::Simple::TranscodeDumb \
--module=List::MoreUtils::XS \
--module=List::SomeUtils::XS \
--module=List::MoreUtils::PP \
--module=HTTP::Status \
--module=HTTP::Date \
--module=Encode:: \
--module=File::Find::Rule \
--module=IO::Socket::SSL \
--module=IO::String \
--module=PerlIO::utf8_strict \
--module=Text::CSV_XS \
--module=DateTime \
Any concrete suggestions as to what I should add to satisfy the required dependencies?
Those are perl module dependencies and as long as you are using the same list as the one in the standard "core" platforms build.sh scripts (Windows, Linux, MacOS), you will be fine. It's the --link lines that pull in the libraries that these modules depend on that matter - this varies from platform to platform and I basically determined these by building a biber with the script, copying it to a VM with no biber or perl on it and running it - if it complains about a missing library, add it to the --link options in the build script. You can get a head start by using the various linker programs like ldd on linux, otool on MacOS and depends.exe on Windows to find out which libraries a binary or perl binary module expects to find.
Some further investigation showed that I have used the wrong voidlinux Docker image for my test. With the correct image, biber extracts and runs in an environment that has neither biber nor perl installed.
$ docker run -ti --rm -v $(pwd)/biber:/usr/local/bin/biber ghcr.io/void-linux/void-linux:20210220rc01-thin-x86_64-musl /bin/sh
# /usr/local/bin/biber
Usage:
biber [options] file[.bcf]
biber [options] --tool <datasource>
Creates "file.bbl" using control file "file.bcf" (".bcf" extension is
optional). Normally use with biblatex requires no options as they are
all set in biblatex and passed via the ".bcf" file
In "tool" mode (see B<--tool> option), takes a datasource (defaults to
"bibtex" datasource) and outputs a copy of the datasource with any command-line
or config file options applied.
Please run "biber --help" for option details
Using the Docker images provided at https://github.com/void-linux/void-docker, I cannot install any software. xbps fails due to certificate errrors:
# xbps-install -v perl
Found perl-5.32.1_2 in repository https://alpha.de.repo.voidlinux.org/current/musl
Found less-563_1 in repository https://alpha.de.repo.voidlinux.org/current/musl
Found gdbm-1.19_1 in repository https://alpha.de.repo.voidlinux.org/current/musl
Name Action Version New version Download size
less install - 563_1 112KB
gdbm install - 1.19_1 139KB
perl install - 5.32.1_2 14MB
Size to download: 14MB
Size required on disk: 69MB
Space available on disk: 67GB
Do you want to continue? [Y/n] Y
[*] Downloading packages
Certificate verification failed for /O=Digital Signature Trust Co./CN=DST Root CA X3
Certificate verification failed for /O=Digital Signature Trust Co./CN=DST Root CA X3
Certificate verification failed for /O=Digital Signature Trust Co./CN=DST Root CA X3
Certificate verification failed for /O=Digital Signature Trust Co./CN=DST Root CA X3
Certificate verification failed for /O=Digital Signature Trust Co./CN=DST Root CA X3
SSL_connect returned 1
ERROR: [trans] failed to download `less-563_1' signature from `https://alpha.de.repo.voidlinux.org/current/musl': Operation not permitted
Transaction failed! see above for errors.
However, if Perl 5.32 is the latest version provided by Void Linux, this might be an issue. Current versions of biber are built with Perl 5.36. It seems that the Docker image is rather old. Unfortunately, I don't have the resources at hand to install a full VM, sorry.
If bibert runs completely without perl installed, installing perl shouldn't make a difference. It's only in the case of missing libs in biber that it might pick up incompatible ones from the OS.
@dkwo That's what I tried first, just didn't copy the output. xbps-install -Su didn't work either with the same cert validation error. I suppose the repository server alpha.de.repo.voidlinux.org has a certificate that can't be validated by the (older) PKI that is baked into the Docker image. As I said, this image is pretty old (more than two years), but it's the latest I could find.
it's likely a timezone issue.
could you try export SSL_NO_VERIFY_PEER=1 and then the update commands?
you should be able to do xbps-install -u xbps and then xbps-install -Su.
Confirmed. With perl-5.36.0_1.x86_64-musl installed I see the same error as in your original post:
# /usr/local/bin/biber
IO.c: loadable library and perl binaries are mismatched (got first handshake key 0xe580080, needed 0xeb80080)
However, running biber without a native perl installation works:
# xbps-install -Su
[*] Updating repository `https://repo-default.voidlinux.org/current/musl/x86_64-musl-repodata' ...
[…]
44 downloaded, 2 installed, 42 updated, 44 configured, 0 removed.
# biber
Usage:
biber [options] file[.bcf]
biber [options] --tool <datasource>
Creates "file.bbl" using control file "file.bcf" (".bcf" extension is
optional). Normally use with biblatex requires no options as they are
all set in biblatex and passed via the ".bcf" file
In "tool" mode (see B<--tool> option), takes a datasource (defaults to
"bibtex" datasource) and outputs a copy of the datasource with any command-line
or config file options applied.
Please run "biber --help" for option details
# xbps-install perl
Name Action Version New version Download size
gdbm install - 1.23_1 167KB
less install - 608_1 127KB
perl install - 5.36.0_1 14MB
Size to download: 14MB
Size required on disk: 72MB
Space available on disk: 66GB
Do you want to continue? [Y/n] y
[*] Downloading packages
gdbm-1.23_1.x86_64-musl.xbps.sig: 512B [avg rate: 16MB/s]
gdbm-1.23_1.x86_64-musl.xbps: 167KB [avg rate: 2325MB/s]
gdbm-1.23_1: verifying RSA signature...
less-608_1.x86_64-musl.xbps.sig: 512B [avg rate: 16MB/s]
less-608_1.x86_64-musl.xbps: 127KB [avg rate: 1827MB/s]
less-608_1: verifying RSA signature...
perl-5.36.0_1.x86_64-musl.xbps.sig: 512B [avg rate: 14MB/s]
perl-5.36.0_1.x86_64-musl.xbps: 14MB [avg rate: 3484KB/s]
perl-5.36.0_1: verifying RSA signature...
[*] Collecting package files
gdbm-1.23_1: collecting files...
less-608_1: collecting files...
perl-5.36.0_1: collecting files...
[*] Unpacking packages
gdbm-1.23_1: unpacking ...
less-608_1: unpacking ...
perl-5.36.0_1: unpacking ...
[*] Configuring unpacked packages
gdbm-1.23_1: configuring ...
gdbm-1.23_1: installed successfully.
less-608_1: configuring ...
less-608_1: installed successfully.
perl-5.36.0_1: configuring ...
perl-5.36.0_1: installed successfully.
3 downloaded, 3 installed, 0 updated, 3 configured, 0 removed.
# biber
IO.c: loadable library and perl binaries are mismatched (got first handshake key 0xe580080, needed 0xeb80080)
Then something to check is the library loading paths. Have a look at the notes in:
https://github.com/plk/biber/blob/dev/dist/linux_x86_64/build.sh
Sometimes the perl libs have RPATH set and you have to strip this out to make sure nothing is looking outside of the biber unpack tree.
Thanks, @plk, that did it. After removing the RPATH from three libraries in the /usr/local/lib/perl5/site_perl/auto/ directory on my build container I was able to run biber in an updated Void Linux container with a local Perl installation. This should match @dkwo's configuration.
The updated binary is uploaded to SourceForge and a revised package has been submitted to CTAN.
@dkwo: Can you verify that the updated binary at https://sourceforge.net/projects/biblatex-biber/files/biblatex-biber/current/binaries/Linux-musl/biber-linux-musl_x86_64.tar.gz/download works for you?
Thank you both, it now works as expected.