Fix R installation errors related to gfortran
mkoeppe opened this issue · 37 comments
As reported
- for 9.0.rc1 in https://groups.google.com/d/msg/sage-release/YaedKjwMhzc/d44U0MM3BwAJ
- for 9.1.beta3 in https://groups.google.com/d/msg/sage-release/xxEfql2Isgg/bV-FQN8VFQAJ
- for 9.1.beta8 in https://groups.google.com/d/msg/sage-release/eMQIlhglN98/bTwDGD9HBQAJ
https://developer.r-project.org/Blog/public/2019/05/15/gfortran-issues-with-lapack/index.html
Upstream: Not yet reported upstream; Will do shortly.
CC: @dimpase @EmmanuelCharpentier @kiwifb @orlitzky @timokau
Component: packages: standard
Author: Matthias Koeppe
Branch/Commit: 5fbea80
Reviewer: Dima Pasechnik
Issue created by migration from https://trac.sagemath.org/ticket/29170
gcc fix here (linked from the R link) https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90329
not clear what they actually did - new option(s) added to gfortran?
Ok, I read that whole thread so no one else has to. Ultimately, this is due to broken code that needs to be updated. The GCC developers have done two things:
- Added
-fc-prototypes-external, to help people fix that broken code. - In the meantime, they have added a workaround in the form of the (on by default)
-ftail-call-workaroundflag (https://gcc.gnu.org/onlinedocs/gfortran/Code-Gen-Options.html). That keeps the broken code working for now, but the option will eventually be turned off and go away.
The workaround was backported all the way back to gcc-7.x, so really what we should be doing is telling everyone to use a new-enough gfortran. The workaround is in gcc-9.2, and based on the release dates, I presume it's also in 7.5, 8.4, and everything newer than that.
We can probably just update gfortran's spkg-configure.m4 to ensure a new-enough version. Maybe for now it would suffice to check if the -ftail-call-workaround flag is supported. That would rule out older compilers from before things were broken, but is easy to implement.
See also: #29378 Update OpenBLAS to 0.3.9 (which also does not fix these errors)
Description changed:
---
+++
@@ -1,6 +1,7 @@
As reported
- for 9.0.rc1 in https://groups.google.com/d/msg/sage-release/YaedKjwMhzc/d44U0MM3BwAJ
- for 9.1.beta3 in https://groups.google.com/d/msg/sage-release/xxEfql2Isgg/bV-FQN8VFQAJ
+- for 9.1.beta8 in https://groups.google.com/d/msg/sage-release/eMQIlhglN98/bTwDGD9HBQAJ
https://developer.r-project.org/Blog/public/2019/05/15/gfortran-issues-with-lapack/index.html
Our R 3.6.2 already contains checks in configure that are intended to address the problem with the Fortran ABI. On one of the failing platforms, however, I see:
[r-3.6.2.p0] checking if need -fno-optimize-sibling-calls for gfortran... yes
[r-3.6.2.p0] checking for type of 'hidden' Fortran character lengths...
[r-3.6.2.p0] checking for xmkmf... no
which looks suspicious (note - no result shown for the type)
from config.log there:
configure:20320: checking for type of 'hidden' Fortran character lengths
/usr/bin/ld: conftestf.o: relocation R_X86_64_32 against `.rodata' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/ld: final link failed: nonrepresentable section on output
collect2: error: ld returned 1 exit status
configure:20384: result:
bug in R's ./configure ?
In any case missing error handling. I'm on it
Adding -fPIC to CFLAGS & FCFLAGS seems to do the trick.
Author: Matthias Koeppe
New commits:
5fbea80 | build/pkgs/r/spkg-install.in: Work around a failing R configure check for hidden Fortran character lengths |
Tests run at https://github.com/mkoeppe/sage/actions/runs/60562504
One needs to check whether the latest R still has this issue, and notify upstream, if it does.
see above - upgrading to the latest release 3.6.3 did not fix the problem. I'll check for unreleased fixes
Also in R trunk there is no error handling.
https://github.com/wch/r-source/blob/trunk/configure.ac#L1108
In this situation, the macro R_PROG_FC_CHAR_LEN_T (from m4/R.m4) produces an empty $r_cv_prog_fc_char_len_t, which causes the compilation errors. Error checking should be added either in the macro or in the macro call.
Upstream does not have an open bug reporting system. Perhaps an R user can communicate this problem to upstream.
Upstream: Not yet reported upstream; Will do shortly.
Shouldn't R also handle the need to add -fPIC in its ./configure ?
I don't really understand what's happening there. It may have to do with the way we compile the gfortran spkg. Note all of the platforms where the failure occured use system gcc but a gfortran from our spkg.
maybe our gfortran doesn't have the needed workaround in, or perhaps there is a gfortran's default that needs to be turned on at gfortran's build time?
Maybe we should get rid of our gfortran instead of R, at least on the various linux distributions where there are other sane alternatives.
there are unfortunate souls that need Sage running on HPC systems, which often have quite outdated software and non-responsive or just absent sysadmins, and so they need to build Sage's toolchain.
I'd spin out the toolchain in a separate package.
I think this is a working fix. I would not want to get rid of the sage distribution's ability to install gfortran. Lots of poorly maintained Linux boxes in universities everywhere.
And we already advertise the distribution packages for gfortran and R.
Tests run at https://github.com/mkoeppe/sage/actions/runs/60588138
This seems to fix the problem. See for example for ubuntu-bionic-minimal at https://github.com/mkoeppe/sage/runs/524868489
Needs review
Description changed:
---
+++
@@ -1,8 +1,15 @@
+The R spkg does not build from source on many major platforms including ubuntu-bionic-minimal, ubuntu-eoan/focal-minimal, debian-buster/bullseye/sid-minimal, linuxmint-19.3-minimal, archlinux-latest-minimal.
+
+On these systems, we use a system gcc with our gfortran 9.2.0.
+
As reported
- for 9.0.rc1 in https://groups.google.com/d/msg/sage-release/YaedKjwMhzc/d44U0MM3BwAJ
- for 9.1.beta3 in https://groups.google.com/d/msg/sage-release/xxEfql2Isgg/bV-FQN8VFQAJ
- for 9.1.beta8 in https://groups.google.com/d/msg/sage-release/eMQIlhglN98/bTwDGD9HBQAJ
-https://developer.r-project.org/Blog/public/2019/05/15/gfortran-issues-with-lapack/index.html
+The present ticket works around a failure in R's configure script.
+
+Related:
+- https://developer.r-project.org/Blog/public/2019/05/15/gfortran-issues-with-lapack/index.html
Reviewer: Dima Pasechnik
Description changed:
---
+++
@@ -1,15 +1,8 @@
-The R spkg does not build from source on many major platforms including ubuntu-bionic-minimal, ubuntu-eoan/focal-minimal, debian-buster/bullseye/sid-minimal, linuxmint-19.3-minimal, archlinux-latest-minimal.
-
-On these systems, we use a system gcc with our gfortran 9.2.0.
-
As reported
- for 9.0.rc1 in https://groups.google.com/d/msg/sage-release/YaedKjwMhzc/d44U0MM3BwAJ
- for 9.1.beta3 in https://groups.google.com/d/msg/sage-release/xxEfql2Isgg/bV-FQN8VFQAJ
- for 9.1.beta8 in https://groups.google.com/d/msg/sage-release/eMQIlhglN98/bTwDGD9HBQAJ
-The present ticket works around a failure in R's configure script.
-
-Related:
-- https://developer.r-project.org/Blog/public/2019/05/15/gfortran-issues-with-lapack/index.html
+https://developer.r-project.org/Blog/public/2019/05/15/gfortran-issues-with-lapack/index.html
ok, it works.
Thanks!
R_PROG_FC_CHAR_LEN_T might be buggy, it does not use any "normal" autoconf macros for test programs, doing instead pure shell.
Replying to @dimpase:
R_PROG_FC_CHAR_LEN_Tmight be buggy, it does not use any "normal" autoconf macros for test programs, doing instead pure shell.
I think you're right and we would wind up with -fPIC added for this one check if they had used the autotools magic. Adding -fPIC globally works around the issue, but isn't a good long-term solution (Case 3 in https://wiki.gentoo.org/wiki/Project:AMD64/Fixing_-fPIC_Errors_Guide).
Since the compile/link command is hard-coded into R_PROG_FC_CHAR_LEN_T, can we append -fPIC for just the one check? Then upstream should be able to either do the same thing, or use some autotools stuff to fix it in the future.
Changed branch from u/mkoeppe/fix_r_installation_errors_related_to_gfortran__or_downgrade_r_to__experimental_ to 5fbea80