Use system openblas also on Fedora-27
mkoeppe opened this issue · 11 comments
mkoeppe commented
Original ticket description
(a) Fedora does not install openblas.pc, so spkg-configure.m4 does not find it.
---> Resolved by #29398 **except for `fedora-27` (see discussion)**
(b) Also, (as noted in #29361) when installing our openblas on fedora-32-standard (which has openblas installed), openblas-dependent packages such as iml, gsl fail to compile. This is in contrast to fedora-32-minimal (which has no system openblas) - where iml builds successfully
---> Resolved by #29378
Depends on #29398
CC: @dimpase
Component: build: configure
Issue created by migration from https://trac.sagemath.org/ticket/29393
mkoeppe commented
comment:1
On fedora-32-standard:
[root@36039437129f sage]# ls /usr/lib64/pkgconfig/
atomic_ops.pc gmp.pc libcrypt.pc libpng.pc libxcrypt.pc panelw.pc xcb-dpms.pc xcb-xinerama.pc
bzip2.pc gmpxx.pc libcurl.pc libpng16.pc libzmq.pc pthread-stubs.pc xcb-dri2.pc xcb-xinput.pc
com_err.pc gsl.pc libffi.pc libselinux.pc m4ri.pc qd.pc xcb-dri3.pc xcb-xkb.pc
eclib.pc gssrpc.pc libjpeg.pc libsepol.pc m4rie.pc readline.pc xcb-glx.pc xcb-xselinux.pc
expat.pc icu-i18n.pc libkeyutils.pc libsodium.pc menu.pc sqlite3.pc xcb-present.pc xcb-xtest.pc
fflas-ffpack.pc icu-io.pc liblzma.pc libtiff-4.pc menuw.pc tcl.pc xcb-randr.pc xcb-xv.pc
fontconfig.pc icu-uc.pc libpcre.pc libunwind-coredump.pc mit-krb5-gssapi.pc tic.pc xcb-record.pc xcb-xvmc.pc
fontutil.pc imagequant.pc libpcre16.pc libunwind-generic.pc mit-krb5.pc tinfo.pc xcb-render.pc xcb.pc
form.pc kadm-client.pc libpcre2-16.pc libunwind-ptrace.pc mpfr.pc tk.pc xcb-res.pc xft.pc
formw.pc kadm-server.pc libpcre2-32.pc libunwind.pc ncurses++.pc tre.pc xcb-screensaver.pc xpm.pc
fplll.pc kdb.pc libpcre2-8.pc libverto.pc ncurses++w.pc x11-xcb.pc xcb-shape.pc xrender.pc
freetype2.pc krb5-gssapi.pc libpcre2-posix.pc libwebp.pc ncurses.pc x11.pc xcb-shm.pc zlib.pc
gdlib.pc krb5.pc libpcre32.pc libwebpdecoder.pc ncursesw.pc xau.pc xcb-sync.pc
gf2x.pc libR.pc libpcrecpp.pc libwebpdemux.pc openpgm-5.2.pc xcb-composite.pc xcb-xf86dri.pc
givaro.pc libRmath.pc libpcreposix.pc libwebpmux.pc panel.pc xcb-damage.pc xcb-xfixes.pc
[root@36039437129f sage]# ls /usr/lib64/*blas*
/usr/lib64/libgslcblas.so /usr/lib64/libopenblas64.so.0 /usr/lib64/libopenblaso64.so /usr/lib64/libopenblasp64-r0.3.9.so
/usr/lib64/libgslcblas.so.0 /usr/lib64/libopenblas64_-r0.3.9.so /usr/lib64/libopenblaso64.so.0 /usr/lib64/libopenblasp64.so
/usr/lib64/libgslcblas.so.0.0.0 /usr/lib64/libopenblas64_.so /usr/lib64/libopenblaso64_-r0.3.9.so /usr/lib64/libopenblasp64.so.0
/usr/lib64/libopenblas-r0.3.9.so /usr/lib64/libopenblas64_.so.0 /usr/lib64/libopenblaso64_.so /usr/lib64/libopenblasp64_-r0.3.9.so
/usr/lib64/libopenblas.so /usr/lib64/libopenblaso-r0.3.9.so /usr/lib64/libopenblaso64_.so.0 /usr/lib64/libopenblasp64_.so
/usr/lib64/libopenblas.so.0 /usr/lib64/libopenblaso.so /usr/lib64/libopenblasp-r0.3.9.so /usr/lib64/libopenblasp64_.so.0
/usr/lib64/libopenblas64-r0.3.9.so /usr/lib64/libopenblaso.so.0 /usr/lib64/libopenblasp.so
/usr/lib64/libopenblas64.so /usr/lib64/libopenblaso64-r0.3.9.so /usr/lib64/libopenblasp.so.0
[root@36039437129f sage]# ls /usr/lib64/*lapack*
ls: cannot access '/usr/lib64/*lapack*': No such file or directory
[root@36039437129f sage]# ls /usr/lib64/*cblas*
/usr/lib64/libgslcblas.so /usr/lib64/libgslcblas.so.0 /usr/lib64/libgslcblas.so.0.0.0
mkoeppe commented
comment:2
iml build error on fedora-32-standard (after building our own openblas):
configure:9370: checking for CBLAS
configure:9406: gcc -o conftest -I. -O3 -g -I/sage/local/include -I///usr/include -L/sage/local/lib -Wl,-rpath,/sage/local/lib -lm conftest.c -L/sage/local/lib -lopenblas -L///usr/lib -lgmp -L/sage/local/lib -lgmp >&5
/usr/bin/ld: /sage/local/lib/libopenblas.so: undefined reference to `_gfortran_concat_string'
collect2: error: ld returned 1 exit status
configure:9406: $? = 1
In fact, our openblas is underlinked in this configuration:
[root@36039437129f sage]# cat local/lib/pkgconfig/openblas.pc
libdir=/sage/local/lib
includedir=/sage/local/include
openblas_config= USE_64BITINT= DYNAMIC_ARCH= DYNAMIC_OLDER= NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= HASWELL MAX_THREADS=10
version=0.3.6
extralib=-lm -lpthread -lm -lpthread
Name: openblas
Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version
Version: ${version}
URL: https://github.com/xianyi/OpenBLAS
Libs: -L${libdir} -lopenblas
Libs.private: ${extralib}
Cflags: -I${includedir}
[root@36039437129f sage]# ./sage -buildsh
(sage-buildsh) root@36039437129f:sage$ echo $LDFLAGS
-L/sage/local/lib -Wl,-rpath,/sage/local/lib
(sage-buildsh) root@36039437129f:sage$ ldd local/lib/libopenblas.so
linux-vdso.so.1 (0x00007ffe58d59000)
libm.so.6 => /lib64/libm.so.6 (0x00007f0f610b8000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0f61096000)
libc.so.6 => /lib64/libc.so.6 (0x00007f0f60ecc000)
/lib64/ld-linux-x86-64.so.2 (0x00007f0f61f11000)
(sage-buildsh) root@36039437129f:sage$ which gfortran
/usr/bin/gfortran
(sage-buildsh) root@36039437129f:sage$ ls -l /usr/lib64/lib*fortran*
lrwxrwxrwx 1 root root 20 Mar 12 00:58 /usr/lib64/libgfortran.so.5 -> libgfortran.so.5.0.0
-rwxr-xr-x 1 root root 2905832 Mar 12 01:04 /usr/lib64/libgfortran.so.5.0.0
(sage-buildsh) root@36039437129f:sage$ nm -D local/lib/libopenblas.so | grep concat_string
U _gfortran_concat_string
(sage-buildsh) root@36039437129f:sage$ nm -D /usr/lib64/libgfortran.so.5 | grep concat_string
0000000000278270 T _gfortran_concat_string
00000000002789e0 T _gfortran_concat_string_char4
mkoeppe commented
comment:3
The underlinking is fixed in openblas 0.3.9 (#29378):
[root@a28d653960b5 sage]# cat local/lib/pkgconfig/openblas.pc
libdir=/sage/local/lib
includedir=/sage/local/include
openblas_config= USE_64BITINT= DYNAMIC_ARCH= DYNAMIC_OLDER= NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= HASWELL MAX_THREADS=10
version=0.3.9
extralib=-lm -lpthread -lgfortran -lm -lpthread -lgfortran
Name: openblas
Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version
Version: ${version}
URL: https://github.com/xianyi/OpenBLAS
Libs: -L${libdir} -lopenblas
Libs.private: ${extralib}
Cflags: -I${includedir}
mkoeppe commented
comment:4
To use system blas, should use AC_SEARCH_LIBS and generate .pc files. This can probably be done in one ticket for both cygwin and fedora-32. See #29398.
mkoeppe commented
comment:5
With #29398, detects openblas correctly on fedora-28-standard, fedora-30-standard (https://github.com/mkoeppe/sage/runs/553984214), fedora-31-standard (https://github.com/mkoeppe/sage/runs/553984224)
checking whether SageMath should install SPKG openblas...
checking BLAS library... openblas
checking whether any of gfortran is installed as or will be installed as SPKG... no
checking for OPENBLAS... no
checking for library containing openblas_get_config... -lopenblas
checking whether openblas_get_config indicates version >= 0.2.20... yes
checking for library containing cblas_dgemm... none required
checking for library containing dgeqrf... no
checking for library containing dgeqrf_... none required
checking for library containing DGEQRF... no
checking for library containing DGEQRF_... no
configure: will use system package and not install SPKG openblas
-----------------------------------------------------------------------------
On fedora-26-standard (https://github.com/mkoeppe/sage/runs/553984168), it correctly rejects openblas 0.2.19
checking for library containing openblas_get_config... -lopenblas
checking whether openblas_get_config indicates version >= 0.2.20... no
checking for library containing cblas_dgemm... none required
checking for library containing dgeqrf... no
checking for library containing dgeqrf_... none required
checking for library containing DGEQRF... no
checking for library containing DGEQRF_... no
configure: no suitable system package found for SPKG openblas
However, fedora-27 has 0.2.20; but version info is not part of openblas_get_config for versions pre 0.3.4.
So the code rejects it. https://github.com/mkoeppe/sage/runs/553984180
mkoeppe commented
comment:7
Is this good enough?
dimpase commented
comment:8
As I always said, past-EOL systems should not be our priority.
mkoeppe commented
Description changed:
---
+++
@@ -1,4 +1,9 @@
-Fedora does not install openblas.pc, so spkg-configure.m4 does not find it
+Original ticket description
-Also, (as noted in #29361) when installing our openblas on fedora-32-standard (which has openblas installed), openblas-dependent packages such as iml, gsl fail to compile. This is in contrast to fedora-32-minimal (which has no system openblas) - where iml builds successfully
+(a) Fedora does not install openblas.pc, so spkg-configure.m4 does not find it.
+ ---> Resolved by #29398 **except for `fedora-27` (see discussion)**
+
+(b) Also, (as noted in #29361) when installing our openblas on fedora-32-standard (which has openblas installed), openblas-dependent packages such as iml, gsl fail to compile. This is in contrast to fedora-32-minimal (which has no system openblas) - where iml builds successfully
+
+ ---> Resolved by #29378mkoeppe commented
comment:9
OK, I'll mark it sage-wishlist then.
mkoeppe commented
outdated