Certain binaries fail to start with "cannot execute binary file" error since UEK 5.4.17-2136.304.4.5
Closed this issue · 12 comments
There seems to be a regression between UEK 5.4.17-2136.304.4.5 and the previous 5.4.17-2102.201.3 version on OL7 with regards to running third-party binaries that are linked against different glibc versions than the one included in the OS.
After updating to this version, running certain binaries fails with a cannot execute binary file error. I was able to reproduce this with the binary tarball distribution of Percona Server for MySQL, as well as a snap install of MicroK8S (as reported on the K8S forum).
I initially reported this issue here, but wanted to make sure this is getting sufficient visibility. Here's a quick example on how to reproduce this:
Boot OL7 using linux-uek-5.4.17-2102.201.3, download Percona Server for MySQL and note that the mysql binary works as expected:
[vagrant@localhost ~]$ cat /etc/oracle-release
Oracle Linux Server release 7.9
[vagrant@localhost ~]$ uname -r
5.4.17-2102.201.3.el7uek.x86_64
[vagrant@localhost ~]$ wget -q https://downloads.percona.com/downloads/Percona-Server-5.7/Percona-Server-5.7.36-39/binary/tarball/Percona-Server-5.7.36-39-Linux.x86_64.glibc2.12.tar.gz.sha256sum
[vagrant@localhost ~]$ wget -q https://downloads.percona.com/downloads/Percona-Server-5.7/Percona-Server-5.7.36-39/binary/tarball/Percona-Server-5.7.36-39-Linux.x86_64.glibc2.12.tar.gz
[vagrant@localhost ~]$ sha256sum -c Percona-Server-5.7.36-39-Linux.x86_64.glibc2.12.tar.gz.sha256sum
Percona-Server-5.7.36-39-Linux.x86_64.glibc2.12.tar.gz: OK
[vagrant@localhost ~]$ tar xf Percona-Server-5.7.36-39-Linux.x86_64.glibc2.12.tar.gz
[vagrant@localhost ~]$ ./Percona-Server-5.7.36-39-Linux.x86_64.glibc2.12/bin/mysql --version
./Percona-Server-5.7.36-39-Linux.x86_64.glibc2.12/bin/mysql Ver 14.14 Distrib 5.7.36-39, for Linux (x86_64) using 6.0
Now perform a yum update and reboot into the 5.4.17-2136.304.4.5 kernel:
[vagrant@localhost ~]$ uname -r
5.4.17-2136.304.4.5.el7uek.x86_64
[vagrant@localhost ~]$ ./Percona-Server-5.7.36-39-Linux.x86_64.glibc2.12/bin/mysql --version
-bash: ./Percona-Server-5.7.36-39-Linux.x86_64.glibc2.12/bin/mysql: cannot execute binary file
Note that this error only seems to occur with third-party binaries that are linked against a different glibc version (as it's the case for the Percona Server binary and the MicroK8S snap). The rest of the OS seems to be working fine.
@LenzGr 👋 Thanks for reporting. We are going to take a look
Root cause identified. Fix in release process.
Hello @totalamateurhour , we experienced the same issue on OL8 while installing microk8S with at least grep and gawk binaries. Your fix will target both OL7 and 8 versions ?
Works: 5.4.17-2102.201.3.el8uek
Fail: 5.4.17-2136.302.7.2.2.el8uek
Thanks for the fix.
@NicolasPrivat Yes, the fix will be for UEK6 (5.4.x), which we provide for both OL7 and OL8.
@LenzGr @NicolasPrivat these are now on yum.oracle.com for x86_64
kernel-uek-5.4.17-2136.305.5.4.el7uek.x86_64
kernel-uek-5.4.17-2136.305.5.4.el8uek.x86_64
Fix confirmed for microk8s (1.22 & 1.23) on kernel-uek-5.4.17-2136.306.1.3.el8uek.x86_64: it now installs (and run) like a charm.
Thanks!
Hi,
Got the same error, upgraded the kernel-uek package to 5.4.17-2136.306.1.3.el8uek.x86_64, and it works fine 😺
OS : Oracle Linux 8.5
microk8s : 1.24
Thanks !
Still having an issue, although it is arm64 not x86_64
# uname -a
Linux oci-arm-dev 5.4.17-2136.307.3.5.el8uek.aarch64 #2 SMP Thu May 26 18:21:01 PDT 2022 aarch64 aarch64 aarch64 GNU/Linux
# grep PRETTY_NAME /etc/os-release
PRETTY_NAME="Oracle Linux Server 8.6"
# snap install microk8s --classic
error: cannot perform the following tasks:
- Run install hook of "microk8s" snap if present (run hook "install":
-----
++ /snap/microk8s/3278/bin/uname -m
+ ARCH=' aarch64'
+ export 'LD_LIBRARY_PATH=/snap/microk8s/3278/lib:/snap/microk8s/3278/usr/lib:/snap/microk8s/3278/lib/ aarch64-linux-gnu:/snap/microk8s/3278/usr/lib/ aarch64-linux-gnu'
+ LD_LIBRARY_PATH='/snap/microk8s/3278/lib:/snap/microk8s/3278/usr/lib:/snap/microk8s/3278/lib/ aarch64-linux-gnu:/snap/microk8s/3278/usr/lib/ aarch64-linux-gnu'
+ export PATH=/snap/microk8s/3278/usr/sbin:/snap/microk8s/3278/usr/bin:/snap/microk8s/3278/sbin:/snap/microk8s/3278/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/bin:/usr/local/bin
+ PATH=/snap/microk8s/3278/usr/sbin:/snap/microk8s/3278/usr/bin:/snap/microk8s/3278/sbin:/snap/microk8s/3278/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/bin:/usr/local/bin
+ export OPENSSL_CONF=/snap/microk8s/current/etc/ssl/openssl.cnf
+ OPENSSL_CONF=/snap/microk8s/current/etc/ssl/openssl.cnf
+ source /snap/microk8s/3278/actions/common/utils.sh
++ [[ /var/lib/snapd/snap/microk8s/3278/meta/hooks/install == \/\s\n\a\p\/\m\i\c\r\o\k\8\s\/\3\2\7\8\/\a\c\t\i\o\n\s\/\c\o\m\m\o\n\/\u\t\i\l\s\.\s\h ]]
+ cp -r /snap/microk8s/3278/default-args /var/snap/microk8s/3278/args
cp: with --parents, the destination must be a directory
Try 'cp --help' for more information.
-----)
@maxsatula this looks to be a different issue than reported by @LenzGr. His error message was cannot execute binary file. Do you have reason to believe it's the same issue?
@maxsatula this looks to be a different issue than reported by @LenzGr. His error message was
cannot execute binary file. Do you have reason to believe it's the same issue?
I don't have 100% proof right now, but it seems to be a different manifestation of the same root cause. Note a leading space in the architecture name, looks like an uname executable runs but internally messes up with something.
@totalamateurhour it must be a very very late comment on this thread, but on arm64 it was also same. This aint the recent comment but if you have got it fixed for arm64 arch, kindly ignore.
If anyone on this issue thread is still seeing this on aarch64, please reopen this issue and chime in so we can look into it.