cygwin-standard: sagelib build fails
mkoeppe opened this issue · 77 comments
https://github.com/sagemath/sage/runs/1003662756
[sagelib-9.2.beta9] gcc -shared -Wl,--enable-auto-image-base -L/cygdrive/d/a/sage/sage/local/lib -Wl,-rpath,/cygdrive/d/a/sage/sage/local/lib build/temp.cygwin-3.1.6-x86_64-3.7/build/cythonized/sage/ext/interpreters/wrapper_rr.o -L/cygdrive/d/a/sage/sage/local/lib/python3.7/config -L/usr/lib -lmpfr -lgmp -lpython3.7m -o build/lib.cygwin-3.1.6-x86_64-3.7/sage/ext/interpreters/wrapper_rr.cpython-37m-x86_64-cygwin.dll -lpari
[sagelib-9.2.beta9] g++ -shared -Wl,--enable-auto-image-base -L/cygdrive/d/a/sage/sage/local/lib -Wl,-rpath,/cygdrive/d/a/sage/sage/local/lib build/temp.cygwin-3.1.6-x86_64-3.7/build/cythonized/sage/symbolic/expression.o -L/cygdrive/d/a/sage/sage/local/lib/python3.7/config -L/usr/lib -lgmp -lpynac -lpython3.7m -o build/lib.cygwin-3.1.6-x86_64-3.7/sage/symbolic/expression.cpython-37m-x86_64-cygwin.dll -lpari
[sagelib-9.2.beta9] error: command 'gcc' failed with exit status 1
CC: @embray @frederichan-IMJPRG @sagetrac-parisse @kiwifb @jhpalmieri @dimpase
Component: porting: Cygwin
Author: Matthias Koeppe
Branch/Commit: e673110
Reviewer: Darij Grinberg, Travis Scrimshaw
Issue created by migration from https://trac.sagemath.org/ticket/30396
Without an error message I wouldn't even know how to begin interpreting this. Is it something you can reproduce locally? Did anything change recently in that file?
That's exactly my problem as well. Very consistently in these builds I get this error but no error message.
Local replication of this failure would already be a step forward. (I don't have a Windows machine.)
Realistically that can't be a release blocker then
I have what appears to be the same error. 9.2.beta12, after distclean.
Last lines after what appears to be a full compilation of sagelib:
[sagelib-9.2.beta12] error: command 'g++' failed with exit status 1
[sagelib-9.2.beta12]
[sagelib-9.2.beta12] real 191m8.476s
[sagelib-9.2.beta12] user 176m28.216s
[sagelib-9.2.beta12] sys 5m47.132s
make[4]: *** [Makefile:2161: sagelib-no-deps] Error 1
make[3]: *** [Makefile:2161: /home/skraeling/sage/local/var/lib/sage/installed/sagelib-9.2.beta12] Error 2
make[2]: *** [Makefile:1769: all-build] Error 2
make[2]: Leaving directory '/home/skraeling/sage/build/make'
Full log: https://www.dropbox.com/s/jjmgib33r9z0359/install.log?dl=0
Thanks for checking this.
I see this error:
[sagelib-9.2.beta12] gcc -shared -Wl,--enable-auto-image-base -L/home/skraeling/sage/local/lib -Wl,-rpath,/home/skraeling/sage/local/lib build/temp.cygwin-3.1.6-x86_64-3.6/build/cythonized/sage/misc/sage_ostools.o -L/home/skraeling/sage/local/lib/python3.6/config -L/usr/lib -lpython3.6m -o build/lib.cygwin-3.1.6-x86_64-3.6/sage/misc/sage_ostools.cpython-36m-x86_64-cygwin.dll
[sagelib-9.2.beta12] /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/../../../../x86_64-pc-cygwin/bin/ld: build/temp.cygwin-3.1.6-x86_64-3.6/build/cythonized/sage/misc/sage_ostools.o: in function `__pyx_pf_4sage_4misc_12sage_ostools_5fix_for_ticket_30157':
[sagelib-9.2.beta12] /home/skraeling/sage/build/pkgs/sagelib/src/build/cythonized/sage/misc/sage_ostools.c:4052: undefined reference to `sqlite3_initialize'
[sagelib-9.2.beta12] /home/skraeling/sage/build/pkgs/sagelib/src/build/cythonized/sage/misc/sage_ostools.c:4052:(.text+0xe05): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `sqlite3_initialize'
[sagelib-9.2.beta12] collect2: error: ld returned 1 exit status
Could you rerun make build one more time after this failure and post the log?
Could you also post your config.log?
Here's a possible fix
New commits:
d133f2c | src/sage/misc/sage_ostools.pyx: Add distutils header |
Sadly, doesn't help.
https://www.dropbox.com/s/z2ayo5teqhfqvoa/install-last.log?dl=0
Where do I find config.log?
Interesting. The file is supposedly there but actually isn't. But this reminds me of the fact that I have not used ./configure. Maybe I should try again....
Replying to @darijgr:
Sadly, doesn't help.
It did, actually, now other errors are preventing the build.
[sagelib-9.2.beta12] g++ -shared -Wl,--enable-auto-image-base -L/home/skraeling/sage/local/lib -Wl,-rpath,/home/skraeling/sage/local/lib build/temp.cygwin-3.1.6-x86_64-3.6/build/cythonized/sage/libs/giac/giac.o -L/home/skraeling/sage/local/lib/python3.6/config -L/usr/lib -lgmp -lgiac -lpython3.6m -o build/lib.cygwin-3.1.6-x86_64-3.6/sage/libs/giac/giac.cpython-36m-x86_64-cygwin.dll -lpari
[sagelib-9.2.beta12] /usr/lib/gcc/x86_64-pc-cygwin/9.3.0/../../../../x86_64-pc-cygwin/bin/ld: /home/skraeling/sage/local/lib/libgiac.a(moyal.o): in function `giac::Airy_Ai(giac::gen const&, giac::context const*)':
[sagelib-9.2.beta12] /home/skraeling/sage/local/var/tmp/sage/build/giac-1.5.0.87p0/src/src/moyal.cc:243: undefined reference to `gsl_sf_airy_Ai'
[sagelib-9.2.beta12] /home/skraeling/sage/local/var/tmp/sage/build/giac-1.5.0.87p0/src/src/moyal.cc:243:(.text+0x63b): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `gsl_sf_airy_Ai'
....
Looks like libgiac is underlinked
Branch pushed to git repo; I updated commit sha1. New commits:
a36d00a | src/sage/libs/giac/giac.pyx: Add libraries to work around underlinked libgiac on cygwin |
Try with this one please
config.log (just ran ./configure): https://www.dropbox.com/s/lv2r3wq2qz55th3/config.log?dl=0
New commits:
a36d00a | src/sage/libs/giac/giac.pyx: Add libraries to work around underlinked libgiac on cygwin |
New failure, with your new commit: https://www.dropbox.com/s/lv2r3wq2qz55th3/config.log?dl=0
Oops, wrong link.
https://www.dropbox.com/s/1lsao5jh7l8i67y/install3.log?dl=0
Branch pushed to git repo; I updated commit sha1. New commits:
d112903 | src/sage_setup/library_order.py: Add giac and dependencies to library_order_list |
Thanks. Here's the next iteration
And more undefined failures...
https://pastebin.com/kGbvs167
Branch pushed to git repo; I updated commit sha1. New commits:
8acaee6 | src/sage/libs/giac/giac.pyx: Add more libraries |
Another round...
Branch pushed to git repo; I updated commit sha1. New commits:
bf9a650 | src/sage/libs/giac/giac.pyx: Add even more libraries |
Standard arguments suggest finite termination of this process...
Shorter again! https://pastebin.com/i8zMiHZ9
Branch pushed to git repo; I updated commit sha1. New commits:
7c0f550 | src/sage_setup/library_order.py: Move giac before ecm |
Getting closer...
Thank you!!! sagelib has compiled now. Will take some hours until I know if the rest of Sage has compiled, but I don't expect bad surprises at this point.
Reviewer: Darij Grinberg
Author: Matthias Koeppe
I am getting the following failure on Ubuntu:
[sagelib-9.2.beta12] g++ -pthread -shared -Wl,-rpath-link,/home/uqtscrim/sage-build/local/lib -L/home/uqtscrim/sage-build/local/lib -Wl,-rpath,/home/uqtscrim/sage-build/local/lib -L. -Wl,-rpath-link,/home/uqtscrim/sage-build/local/lib -L/home/uqtscrim/sage-build/local/lib -Wl,-rpath,/home/uqtscrim/sage-build/local/lib -Wl,-rpath-link,/home/uqtscrim/sage-build/local/lib -L/home/uqtscrim/sage-build/local/lib -Wl,-rpath,/home/uqtscrim/sage-build/local/lib build/temp.linux-x86_64-3.8/build/cythonized/sage/libs/giac/giac.o -L/home/uqtscrim/sage-build/local/lib -L/home/uqtscrim/sage-build/local/lib -lgiac -lintl -lcurl -lecm -lgsl -lglpk -lmpfi -lmpfr -lntl -lgmp -lm -lopenblas -o build/lib.linux-x86_64-3.8/sage/libs/giac/giac.cpython-38-x86_64-linux-gnu.so -lpari
[sagelib-9.2.beta12] /usr/bin/ld: cannot find -lintl
[sagelib-9.2.beta12] collect2: error: ld returned 1 exit status
[sagelib-9.2.beta12] error: command 'g++' failed with exit status 1
[sagelib-9.2.beta12] Exception ignored in: <function Pool.__del__ at 0x7f17cc3233a0>
[sagelib-9.2.beta12] Traceback (most recent call last):
[sagelib-9.2.beta12] File "/home/uqtscrim/sage-build/local/lib/python3.8/multiprocessing/pool.py", line 268, in __del__
[sagelib-9.2.beta12] self._change_notifier.put(None)
[sagelib-9.2.beta12] File "/home/uqtscrim/sage-build/local/lib/python3.8/multiprocessing/queues.py", line 368, in put
[sagelib-9.2.beta12] self._writer.send_bytes(obj)
[sagelib-9.2.beta12] File "/home/uqtscrim/sage-build/local/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes
[sagelib-9.2.beta12] self._send_bytes(m[offset:offset + size])
[sagelib-9.2.beta12] File "/home/uqtscrim/sage-build/local/lib/python3.8/multiprocessing/connection.py", line 411, in _send_bytes
[sagelib-9.2.beta12] self._send(header + buf)
[sagelib-9.2.beta12] File "/home/uqtscrim/sage-build/local/lib/python3.8/multiprocessing/connection.py", line 368, in _send
[sagelib-9.2.beta12] n = write(self._handle, buf)
[sagelib-9.2.beta12] OSError: [Errno 9] Bad file descriptor
[sagelib-9.2.beta12]
[sagelib-9.2.beta12] real 0m46.175s
[sagelib-9.2.beta12] user 0m42.734s
[sagelib-9.2.beta12] sys 0m1.811s
Makefile:2191: recipe for target 'sagelib-no-deps' failed
make[4]: *** [sagelib-no-deps] Error 1
Makefile:2191: recipe for target '/home/uqtscrim/sage-build/local/var/lib/sage/installed/sagelib-9.2.beta12' failed
make[3]: *** [/home/uqtscrim/sage-build/local/var/lib/sage/installed/sagelib-9.2.beta12] Error 2
Makefile:1799: recipe for target 'all-build' failed
make[2]: *** [all-build] Error 2
make[2]: Leaving directory '/home/uqtscrim/sage-build/build/make'
Something like this was to be expected
Build complete! Thanks again, Matthias.
PS. Any idea why it says this
│ SageMath version 9.2.beta12, Release Date: 2020-09-06 │
│ Using Python 3.6.10. Type "help()" for help. │
when 9.2.beta12 supposedly should be using Python 3.8.5?
Replying to @darijgr:
PS. Any idea why it says this
│ SageMath version 9.2.beta12, Release Date: 2020-09-06 │ │ Using Python 3.6.10. Type "help()" for help. │when 9.2.beta12 supposedly should be using Python 3.8.5?
Sage will try to use your system Python
if it is >= 3.6 and < 3.9.
Or it can build its own Python which will be 3.8.5.
To make sure it builds its own Python
if you build from scratch:
$ make configure
$ ./configure --with-system-python3=no -q
$ make -s V=0
where
- you can replace
make configureby./bootstrap -qis for "quiet", drop it for more verbose output- `-s V=0" are for "silent" and "lowest verbosity",
drop them for more verbose output
To force Sage to rebuild Python and everything that
depends on it, after it was built using the ambient
system's Python:
$ V=0 sage -f python3
where again V=0 is for less verbose output,
drop it if you like more verbose output.
See #30546 for picking the most recent system Python
when several versions are available.
Thank you! I probably won't bother to reinstall Sage right now, but on my next reinstall I'll take this into account. (And probably update cygwin beforehand.)
Attachment: giac-1.5.0.87p1.tar.bz2.gz
Branch pushed to git repo; I updated commit sha1. New commits:
56ebce3 | build/pkgs/giac: Ensure shared library build on cygwin |
Branch pushed to git repo; I updated commit sha1. New commits:
d48a91a | src/sage/libs/giac/giac.pyx: Back out workarounds for library dependencies on cygwin |
Darij, if you have time, another test with this version would be helpful... (This needs ./configure --enable-download-from-upstream-url)
This errors out again again:
https://www.dropbox.com/s/do329gofsaxxw3w/install6.log?dl=0
Note that I have not run "make distclean" after pulling; I've just done the ./configure you suggested and tried to make build again.
Attachment: giac-1.5.0.87p2.tar.bz2.gz
Branch pushed to git repo; I updated commit sha1. New commits:
c47b078 | build/pkgs/giac: Fixup: Ensure shared library build on cygwin |
Thanks for testing. Here's a better version
Changed reviewer from Darij Grinberg to Darij Grinberg, https://github.com/mkoeppe/sage/actions/runs/250810927
Smaller install.log, still failing (don't mind the interrupt; I had to turn off my PC in the process). Again it's sagelib that's failing, not giac.
https://www.dropbox.com/s/xhzrzwzt27643wf/install7.log?dl=0
Thanks for testing, we are getting closer.
[giac-1.5.0.87p2] /usr/bin/bash ../libtool --tag=CXX --mode=link g++ -std=gnu++11 -g -O2 -fno-strict-aliasing -DGIAC_GENERIC_CONSTANTS -no-undefined -L/home/skraeling/sage/local/lib -Wl,-rpath,/home/skraeling/sage/local/lib -o libgiac.la -rpath /home/skraeling/sage/local/lib input_lexer.lo sym2poly.lo gausspol.lo threaded.lo moyal.lo maple.lo ti89.lo mathml.lo misc.lo permu.lo quater.lo desolve.lo input_parser.lo symbolic.lo index.lo modpoly.lo modfactor.lo ezgcd.lo derive.lo solve.lo intg.lo intgab.lo risch.lo lin.lo series.lo subst.lo vecteur.lo sparse.lo csturm.lo tex.lo global.lo ifactor.lo alg_ext.lo gauss.lo isom.lo plot.lo plot3d.lo rpn.lo prog.lo pari.lo cocoa.lo unary.lo usual.lo identificateur.lo gen.lo tinymt32.lo first.lo TmpLESystemSolver.lo TmpFGLM.lo help.lo lpsolve.lo optimization.lo signalprocessing.lo graphe.lo graphtheory.lo nautywrapper.lo markup.lo kdisplay.lo kadd.lo -lntl -lpari -lgsl -lopenblas -lm -lopenblas -lopenblas -lintl -lrt -lpthread -lnauty -lcurl -lglpk -ldl -lm -lecm -lmpfi -lmpfr -lgmp
[giac-1.5.0.87p2]
[giac-1.5.0.87p2] *** Warning: linker path does not have real file for library -lnauty.
[giac-1.5.0.87p2] *** I have the capability to make that library automatically link in when
[giac-1.5.0.87p2] *** you link to this library. But I can only do this if you have a
[giac-1.5.0.87p2] *** shared version of the library, which you do not appear to have
[giac-1.5.0.87p2] *** because I did check the linker path looking for a file starting
[giac-1.5.0.87p2] *** with libnauty and none of the candidates passed a file format test
[giac-1.5.0.87p2] *** using a file magic. Last file checked: /home/skraeling/sage/local/lib/libnauty.a
[giac-1.5.0.87p2] *** The inter-library dependencies that have been dropped here will be
[giac-1.5.0.87p2] *** automatically added whenever a program is linked with this library
[giac-1.5.0.87p2] *** or is declared to -dlopen it.
[giac-1.5.0.87p2]
[giac-1.5.0.87p2] *** Since this library must not contain undefined symbols,
[giac-1.5.0.87p2] *** because either the platform does not support them or
[giac-1.5.0.87p2] *** it was explicitly requested with -no-undefined,
[giac-1.5.0.87p2] *** libtool will only create a static version of it.
[giac-1.5.0.87p2] libtool: link: ar cru .libs/libgiac.a input_lexer.o sym2poly.o gausspol.o threaded.o moyal.o maple.o ti89.o mathml.o misc.o permu.o quater.o desolve.o input_parser.o symbolic.o index.o modpoly.o modfactor.o ezgcd.o derive.o solve.o intg.o intgab.o risch.o lin.o series.o subst.o vecteur.o sparse.o csturm.o tex.o global.o ifactor.o alg_ext.o gauss.o isom.o plot.o plot3d.o rpn.o prog.o pari.o cocoa.o unary.o usual.o identificateur.o gen.o tinymt32.o first.o TmpLESystemSolver.o TmpFGLM.o help.o lpsolve.o optimization.o signalprocessing.o graphe.o graphtheory.o nautywrapper.o markup.o kdisplay.o kadd.o
[giac-1.5.0.87p2] libtool: link: ranlib .libs/libgiac.a
[giac-1.5.0.87p2] libtool: link: ( cd ".libs" && rm -f "libgiac.la" && ln -s "../libgiac.la" "libgiac.la" )
[giac-1.5.0.87p2] /usr/bin/bash ../libtool --tag=CXX --mode=link g++ -std=gnu++11 -g -O2 -fno-strict-aliasing -DGIAC_GENERIC_CONSTANTS -no-undefined -L/home/skraeling/sage/local/lib -Wl,-rpath,/home/skraeling/sage/local/lib -o libxcas.la History.lo Input.lo Xcas1.lo Equation.lo Print.lo Tableur.lo Editeur.lo Graph.lo Graph3d.lo Help1.lo Cfg.lo Flv_CStyle.lo Flve_Check_Button.lo Flve_Input.lo Flv_Style.lo Flv_Data_Source.lo Flve_Combo.lo Flv_List.lo Flv_Table.lo gl2ps.lo libgiac.la -lintl -lrt -lpthread -lnauty -lcurl -lglpk -ldl -lm -lecm -lmpfi -lmpfr -lgmp
Branch pushed to git repo; I updated commit sha1. New commits:
e673110 | build/pkgs/giac/spkg-install.in [Cygwin]: Disable linking with libnauty.a |
Please try this one
Compiles and runs this time. Good job! (That said, I haven't "make distclean"ed.)
Great! Thanks very much for testing.
Changed reviewer from Darij Grinberg, https://github.com/mkoeppe/sage/actions/runs/250810927 to Darij Grinberg
pats laptop on the back
Build from scratch at https://github.com/mkoeppe/sage/runs/1106904182 succeeded in building sagelib too
This now works for me on Ubuntu.
Positive review?
Changed reviewer from Darij Grinberg to Darij Grinberg, Travis Scrimshaw
In terms of things working, yes, but I don't think having new tarballs of a tweaked giac are the way to go. The standard ways are something released from upstream or you include the patch(es) you applied in the $SAGE_ROOT/build/pkgs/giac/patches/ folder with a change in the patch level to 1.5.0.87p1 in the package-version.txt file. (It would also be good to remove the patched bzips from trac to avoid clutter that way.)
The new patch modifies autotools source files and therefore the tarball has to be regenerated. See https://doc.sagemath.org/html/en/developer/packaging.html#when-to-patch-when-to-repackage-when-to-autoconfiscate
I see. Thanks for the explanation.
These are in the nitpick category: Could the patched tarball name have a slightly more descriptive description (the numbering p2 also doesn't make sense once it goes into Sage as well). I would leave it also as .p0 since you are patching that new modified version for the first time.
giac-1.5.0.87p2 identifies the patched tarball uniquely on the Sage mirrors (... and in testers' upstream/ directories)
However, it won't have a good history on the mirrors as it feels arbitrary. Anyways, I don't care that strongly about it, so feel free to set a positive review if you want to keep the name.
Thanks.
Changed branch from u/mkoeppe/cygwin_standard__sagelib_build_fails to e673110