upgrade GAP to 4.11.1 or newer
dimpase opened this issue · 44 comments
the current GAP release is 4.11.1. Time to upgrade - previous upgrade
was in #29314.
Here we also upgrade libsemigroups, in lockstep with GAP, to 1.1.0.
GAP tarball is now hosted on GitHub; we also need to install more GAP packages to satsify needs of updated GAP package digraph and semigroups.
Finally, we fix a bug in TestSuite for abelian groups
CC: @antonio-rojas @embray @kiwifb @mkoeppe @slel @tobihan @videlec
Component: packages: standard
Keywords: upgrade, gap
Author: Dima Pasechnik
Branch: 82f3fa7
Reviewer: Volker Braun, Travis Scrimshaw
Issue created by migration from https://trac.sagemath.org/ticket/31498
Changed keywords from none to upgrade, gap
Author: Dima Pasechnik
Branch: u/dimpase/packages/gap4111
Branch pushed to git repo; I updated commit sha1. New commits:
8367f8e | ctbl versions now numbered |
what remains is to fix this weird error - something is emitting that string of * whcih overrides the test output. Not reproducible at sage: prompt:
File "src/sage/tests/gap_packages.py", line 8, in sage.tests.gap_packages
Failed example:
test_packages(pkgs, only_failures=True) # optional - gap_packages
Expected:
...
Status Package GAP Output
+--------+---------+------------+
Got:
Status Package GAP Output
+--------+---------+------------+
**********************************************************************
but
sage: from sage.tests.gap_packages import all_installed_packages, test_packages
sage: pkgs = all_installed_packages(ignore_dot_gap=True)
sage: test_packages(pkgs, only_failures=True)
Status Package GAP Output
+--------+---------+------------+
sage:
Replying to @dimpase:
what remains is to fix this weird error - something is emitting that string of
*whcih overrides the test output.
Isn't that just the usual test failure output? It seems to me that the actual issue is the "..." wildcard on the first line - there is no newline before the "Status" line so it doesn't match.
Branch pushed to git repo; I updated commit sha1. New commits:
7170197 | modify the test so that Sage's "table" is not used |
another test to fix:
sage -t --warn-long 50.1 --random-seed=0 src/sage/groups/abelian_gps/abelian_group_gap.py
**********************************************************************
File "src/sage/groups/abelian_gps/abelian_group_gap.py", line 201, in sage.groups.abelian_gps.abelian_group_gap.AbelianGroupElement_polycyclic
Failed example:
TestSuite(G.an_element()).run() # optional - gap_packages
Expected nothing
Got:
Failure in _test_pickling:
Traceback (most recent call last):
File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/misc/sage_unittest.py", line 297, in run
test_method(tester=tester)
File "sage/structure/sage_object.pyx", line 647, in sage.structure.sage_object.SageObject._test_pickling (build/cythonized/sage/structure/sage_object.c:5023)
tester.assertEqual(loads(dumps(self)), self)
File "sage/misc/persist.pyx", line 935, in sage.misc.persist.loads (build/cythonized/sage/misc/persist.c:7536)
return unpickler.load()
File "sage/structure/parent.pyx", line 898, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9335)
return mor._call_(x)
File "sage/structure/coerce_maps.pyx", line 161, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4622)
raise
File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (build/cythonized/sage/structure/coerce_maps.c:4514)
return C._element_constructor(x)
File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/groups/abelian_gps/abelian_group_gap.py", line 368, in _element_constructor_
x *= gens_gap[i]**(exp[i] % orders[i])
File "sage/rings/integer.pyx", line 3413, in sage.rings.integer.Integer.__mod__ (build/cythonized/sage/rings/integer.c:22616)
raise ZeroDivisionError("Integer modulo by zero")
ZeroDivisionError: Integer modulo by zero
------------------------------------------------------------
The following tests failed: _test_pickling
**********************************************************************
Branch pushed to git repo; I updated commit sha1. New commits:
5c885dd | do not take "mod 0", skip "mod" |
Description changed:
---
+++
@@ -1 +1,8 @@
-the current GAP release is 4.11.1. Time to upgrade.
+the current GAP release is 4.11.1. Time to upgrade - previous upgrade
+was in #29314.
+
+Here we also upgrade libsemigroups, in lockstep with GAP, to 1.1.0.
+
+GAP tarball is now hosted on [GitHub](../wiki/GitHub); we also need to install more GAP packages to satsify needs of updated GAP package digraph and semigroups.
+
+Finally, we fix a bug in TestSuite for abelian groupsplease review...
I won't lie, I am not quite ready for this in sage-on-gentoo. I have been under rather heavy stress so far this year and I am not doing as much as I want.
Tell me about it ;-)
But I think we should have people bang away at this asap and not later in the release cycle.
Thanks!
Reviewer: Volker Braun
There is a small merge conflict with #31489, and a failure
sage -t --long --warn-long 43.1 --random-seed=0 src/sage/tests/gap_packages.py
**********************************************************************
File "src/sage/tests/gap_packages.py", line 9, in sage.tests.gap_packages
Failed example:
for p in pkgs:
pkg = p.split('-')[0]
if not libgap.LoadPackage(pkg):
raise
Exception raised:
Traceback (most recent call last):
File "/home/release/Sage/local/lib64/python3.9/site-packages/sage/doctest/forker.py", line 714, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/home/release/Sage/local/lib64/python3.9/site-packages/sage/doctest/forker.py", line 1133, in compile_and_execute
exec(compiled, globs)
File "<doctest sage.tests.gap_packages[3]>", line 4, in <module>
raise
RuntimeError: No active exception to reraise
**********************************************************************
1 item had failures:
1 of 6 in sage.tests.gap_packages
[12 tests, 1 failure, 0.26 s]
----------------------------------------------------------------------
sage -t --long --warn-long 43.1 --random-seed=0 src/sage/tests/gap_packages.py # 1 doctest failed
----------------------------------------------------------------------
Replying to @vbraun:
There is a small merge conflict with #31489, and a failure
sage -t --long --warn-long 43.1 --random-seed=0 src/sage/tests/gap_packages.py ...
I only see this if I manually uninstall gap_packages. But this meant to be with them installed?
96396e238d4 (Volker Braun 2013-07-22 17:05:24 -0400 2) Test the optional GAP packages
GAP doesn't have a clean way to uninstall packages.
I can rebase over #31489 if it helps.
these test_packages stuff is a mess. We have two different functions
that both claim to Return list of all installed packages., test_packages(), and all_installed_packages() which differ in the output format - the former outputs table (sic!)
and the latter does not check whether these packages actually may be loaded (so it's largely meaningless).
May we do it on another ticket?
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
a9ac506 | GAP version bump to 4.11.1 |
f8c87d4 | ctbl versions now numbered |
a2bc010 | update libsemigroup |
40e6105 | more GAP packages are deps |
8d44ad0 | modify the test so that Sage's "table" is not used |
05b434a | do not take "mod 0", skip "mod" |
7ee7b58 | better fix for the test |
I believe the test is fixed now.
Branch pushed to git repo; I updated commit sha1. New commits:
101bd74 | use the change from #31489 |
OK, now it depends on #31489 and should cleanly merge: the only change from #31489 is done in comment:22
rebased on 9.4.beta1
Let's try to get this into the next beta.
Changed reviewer from Volker Braun to Volker Braun, Travis Scrimshaw
Changed branch from u/dimpase/packages/gap4111 to 82f3fa7
I am a bit dumbfounded by this upgrade in a way. There are strange particulars to it in sage-on-gentoo. I cannot build the doc.
[categorie] /var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/categories/finite_coxeter_groups.py:docstring of sage.categories.finite_coxeter_groups.FiniteCoxeterGroups.ParentMethods.permutahedron:50: WARNING: Exception occurred in plotting finite_coxeter_groups-1
[categorie] from /var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/doc/en/reference/categories/sage/categories/finite_coxeter_groups.rst:
[categorie] Traceback (most recent call last):
[categorie] File "/usr/lib/python3.9/site-packages/matplotlib/sphinxext/plot_directive.py", line 483, in run_code
[categorie] exec(code, ns)
[categorie] File "<string>", line 1, in <module>
[categorie] File "sage/misc/lazy_import.pyx", line 362, in sage.misc.lazy_import.LazyImport.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/lazy_import.c:4041)
[categorie] return self.get_object()(*args, **kwds)
[categorie] File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/combinat/root_system/coxeter_group.py", line 135, in CoxeterGroup
[categorie] return CoxeterMatrixGroup(cartan_type, base_ring, index_set)
[categorie] File "sage/misc/classcall_metaclass.pyx", line 322, in sage.misc.classcall_metaclass.ClasscallMetaclass.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/classcall_metaclass.c:1743)
[categorie] return cls.classcall(cls, *args, **kwds)
[categorie] File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/groups/matrix_gps/coxeter_group.py", line 230, in __classcall_private__
[categorie] return super(CoxeterMatrixGroup, cls).__classcall__(cls,
[categorie] File "sage/misc/cachefunc.pyx", line 1001, in sage.misc.cachefunc.CachedFunction.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/cachefunc.c:6087)
[categorie] w = self.f(*args, **kwds)
[categorie] File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/structure/unique_representation.py", line 1007, in __classcall__
[categorie] instance = typecall(cls, *args, **options)
[categorie] File "sage/misc/classcall_metaclass.pyx", line 486, in sage.misc.classcall_metaclass.typecall (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/classcall_metaclass.c:2207)
[categorie] return (<PyTypeObject*>type).tp_call(cls, args, kwds)
[categorie] File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/groups/matrix_gps/coxeter_group.py", line 273, in __init__
[categorie] one = MS.one()
[categorie] File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/cachefunc.c:12918)
[categorie] self.cache = f(self._instance)
[categorie] File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/matrix/matrix_space.py", line 1608, in identity_matrix
[categorie] A = self.zero_matrix().__copy__()
[categorie] File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/cachefunc.c:12918)
[categorie] self.cache = f(self._instance)
[categorie] File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/matrix/matrix_space.py", line 1766, in zero_matrix
[categorie] zero = self.base_ring().zero()
[categorie] File "sage/misc/cachefunc.pyx", line 2310, in sage.misc.cachefunc.CachedMethodCallerNoArgs.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/misc/cachefunc.c:12918)
[categorie] self.cache = f(self._instance)
[categorie] File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/rings/universal_cyclotomic_field.py", line 1397, in zero
[categorie] return self.element_class(self, libgap.zero())
[categorie] File "sage/libs/gap/libgap.pyx", line 618, in sage.libs.gap.libgap.Gap.zero (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/gap/libgap.c:6277)
[categorie] return self(0)
[categorie] File "sage/structure/parent.pyx", line 898, in sage.structure.parent.Parent.__call__ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/structure/parent.c:9342)
[categorie] return mor._call_(x)
[categorie] File "sage/structure/coerce_maps.pyx", line 161, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/structure/coerce_maps.c:4628)
[categorie] raise
[categorie] File "sage/structure/coerce_maps.pyx", line 156, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/structure/coerce_maps.c:4520)
[categorie] return C._element_constructor(x)
[categorie] File "sage/libs/gap/libgap.pyx", line 298, in sage.libs.gap.libgap.Gap._element_constructor_ (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/gap/libgap.c:3391)
[categorie] initialize()
[categorie] File "sage/libs/gap/util.pyx", line 314, in sage.libs.gap.util.initialize (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/gap/util.c:5897)
[categorie] with atomic_write(workspace) as f:
[categorie] File "sage/libs/gap/util.pyx", line 316, in sage.libs.gap.util.initialize (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/gap/util.c:5852)
[categorie] gap_eval('SaveWorkspace("{0}")'.format(f.name))
[categorie] File "sage/libs/gap/util.pyx", line 398, in sage.libs.gap.util.gap_eval (/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/cythonized/sage/libs/gap/util.c:6068)
[categorie] GAP_Enter()
[categorie] sage.libs.gap.util.GAPError: Error, reached the pre-set memory limit
[categorie] (change it with the -o command line option)
[graphs ] The inventory files are in ../../build_doc/inventory/en/reference/graphs.
Build finished. The built documents can be found in /var/tmp/portage/sci-mathematics/sage-9999/work/build_doc/inventory/en/reference/graphs
[functions] building [inventory]: targets for 20 source files that are out of date
[functions] updating environment: [new config] 20 added, 0 changed, 0 removed
[finite_ri] The inventory files are in ../../build_doc/inventory/en/reference/finite_rings.
Build finished. The built documents can be found in /var/tmp/portage/sci-mathematics/sage-9999/work/build_doc/inventory/en/reference/finite_rings
[function_] The inventory files are in ../../build_doc/inventory/en/reference/function_fields.
Build finished. The built documents can be found in /var/tmp/portage/sci-mathematics/sage-9999/work/build_doc/inventory/en/reference/function_fields
[game_theo] building [inventory]: targets for 6 source files that are out of date
[game_theo] updating environment: [new config] 6 added, 0 changed, 0 removed
[games ] building [inventory]: targets for 5 source files that are out of date
[games ] updating environment: [new config] 5 added, 0 changed, 0 removed
[game_theo] The inventory files are in ../../build_doc/inventory/en/reference/game_theory.
Build finished. The built documents can be found in /var/tmp/portage/sci-mathematics/sage-9999/work/build_doc/inventory/en/reference/game_theory
[categorie] The inventory files are in ../../build_doc/inventory/en/reference/categories.
Error building the documentation.
Traceback (most recent call last):
File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__main__.py", line 2, in <module>
main()
File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 1813, in main
builder()
File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 345, in _wrapper
getattr(get_builder(document), 'inventory')(*args, **kwds)
File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 582, in _wrapper
self._build_everything_except_bibliography(format, *args, **kwds)
File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 565, in _build_everything_except_bibliography
build_many(build_ref_doc, non_references)
File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/__init__.py", line 297, in build_many
_build_many(target, args, processes=processes)
File "/var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/sage_docbuild/utils.py", line 289, in build_many
raise worker_exc.original_exception
OSError: /var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/build/lib/sage/categories/finite_coxeter_groups.py:docstring of sage.categories.finite_coxeter_groups.FiniteCoxeterGroups.ParentMethods.permutahedron:50: WARNING: Exception occurred in plotting finite_coxeter_groups-1
In sage-on-gentoo, I do separate gap packages and the source of error is more precisely ctbllib-1.3.1+ (I say + because there is a 1.3.2 now in its upstream and the result is the same with it). If I downgrade ctbllib to 1.2_p2 the documentation builds.
The error message suggests to increase the memory limit. This is set somewhere in sage/interface/gap.py and I pushed it from the current 400MB to 2000MB without improvement.
I am probably missing something but what?
Whats the output of _get_gap_memory_pool_size_MB?
GAP needs swap due to how they reserve the memory pool.
Replying to @vbraun:
Whats the output of
_get_gap_memory_pool_size_MB?GAP needs swap due to how they reserve the memory pool.
568MB if I leave it to the default. With the changes for my last attempt, it should be 2001MB. But what puzzles me is the influence of ctbllib. So, I am wondering if the message is in fact unhelpful for the issue at hand.