sagemath/sage

Replace most of sage.libs.gap with gappy

embray opened this issue · 74 comments

From the gappy README:

gappy provides a Python interface to the GAP
computer algebra system by linking to its
library interface.

This is a follow-up to #31297, particularly inspired by #31297 comment:17. It is a more disruptive change, in that instead of providing wrappers around gappy that are Sage Parents and Elements, it follows the example of cypari2 and just uses gappy more-or-less directly without any wrappers.

It remains completely agnostic to the coercion system, though I am not completely happy with this state of affairs. In particular you can see I had to add a special case to Polynomial.__call__ for handling evaluating polynomials on GapObjs, a case that used to work fine, but now needs a special case since other Sage types cannot be coerced to GapObjs.

Also had to add special cases for instantiating Integers and Rationals from GapObjs, but on the plus side this is now a bit faster.

Upstream: Reported upstream. No feedback yet.

CC: @videlec @dimpase @mkoeppe @slel

Component: interfaces

Keywords: gap libgap gappy

Work Issues: release gappy 0.1.0 final and update spkg version before merging, test against p_groups_cohomology

Author: Erik Bray

Branch/Commit: u/dimpase/gappy-without-wrappers @ 3f1d051

Reviewer: Dima Pasechnik, ...

Issue created by migration from https://trac.sagemath.org/ticket/31404

Last 10 new commits:

197290fFix miscellaneous tests that broke due to different group iteration
9f621e9Update gappy to v0.1.0a1 with some fixes for matrices and gap_function.
4839316Miscellaneous updates to not use deprecated interfaces, particularly
baa0f61Update gappy to v0.1.0a2 which adds MacOS and Cygwin fixes.
a07f20eFixes needed for gappy v0.1.0a2 which did away with the libgap_soname
c9160c1Further rework of 45b175e43621795f890204a9cdfb30b524f961fa which
0ae791eUpdate gappy to v0.1.3a3 with the requisite changes needed to adapt to
82e7e56A few minor test fixes:
b305f39Don't assume that anything with a 'parent' attribute is fully
943b009Deprecate sage.libs.gap.element and convert Sage to use

Commit: 943b009

comment:2

If eventually approved, this would supersede #31297.

Description changed:

--- 
+++ 
@@ -1,5 +1,5 @@
 This is a follow-up to #31297, particularly inspired by [#31297 comment:17](https://github.com/sagemath/sage/issues/31297#comment:17).  It is a more disruptive change, in that instead of providing wrappers around gappy that are Sage Parents and Elements, it follows the example of cypari2 and just uses gappy more-or-less directly without any wrappers.
 
-It remains completely agnostic to the coercion system, though I am not completely happy with this state of affairs.  In particular you can see I had to add a special case to `Polynomial.__call__` for handling evaluating polynomials on GapObjs, a case that used to work fine, but now needs a special case since other Sage types cannot be coerced to GapObjs.
+It remains completely agnostic to the coercion system, though I am not completely happy with this state of affairs.  In particular you can see I had to add a special case to `Polynomial.__call__` for handling evaluating polynomials on `GapObj`s, a case that used to work fine, but now needs a special case since other Sage types cannot be coerced to `GapObj`s.
 
 Also had to add special cases for instantiating Integers and Rationals from GapObjs, but on the plus side this is now a bit faster.
comment:3

Does sphinx link such as :mod:`gappy.gapobj` actually works? The question is merely

  • where is it resolved?
  • in the documentation where does it point to: local or distant documentation?

For pplpy we had to trick a bit, (commit 684e10)

diff --git a/build/pkgs/pplpy/spkg-install b/build/pkgs/pplpy/spkg-install
index deba1bb42b..5a39f43925 100644
--- a/build/pkgs/pplpy/spkg-install
+++ b/build/pkgs/pplpy/spkg-install
@@ -1 +1,14 @@
 cd src && sdh_pip_install .
+
+if [[ "$SAGE_SPKG_INSTALL_DOCS" != no ]] ; then
+    # Compile pplpy's documentation
+    sage-python23 setup.py build_ext --inplace
+    PYTHONPATH=$PYTHONPATH:$(pwd)
+    cd docs
+    $MAKE html
+
+    # install pplpy's documentation
+    PPLPY_DOCS=$SAGE_SHARE/doc/pplpy
+    mkdir -p $PPLPY_DOCS
+    cp -r build/html/*  $PPLPY_DOCS
+fi
diff --git a/src/doc/common/conf.py b/src/doc/common/conf.py
index 91f651a41c..70dd17f271 100644
--- a/src/doc/common/conf.py
+++ b/src/doc/common/conf.py
@@ -177,7 +177,8 @@ python_version = sys.version_info.major
 intersphinx_mapping = {
     'python': ('https://docs.python.org/',
                 os.path.join(SAGE_DOC_SRC, "common",
-                             "python{}.inv".format(python_version)))}
+                             "python{}.inv".format(python_version))),
+    'pplpy': (os.path.join(SAGE_SHARE, "doc", "pplpy"), None)}
 
 def set_intersphinx_mappings(app):
     """

What we did is not robust at all...

comment:4

I think the Python package pplpy should advertise the install location of the docs if possible. Assuming that things live in SAGE_SHARE is not a good idea...
See also: #27495 - Use sphinx.ext.extlinks to add Sphinx roles for links to external package docs

comment:5

I am still working on updating the docs.

comment:6

Replying to @mkoeppe:

SNIP

I opened a thread on sage-devel, no need to clutter the ticket with general documentation issue.

comment:7

Another tricky aspect of the doc build: When I try to add a spkg-postinst to install the docs for gappy, gappy's own conf.py has an intersphinx mapping for the Python docs.

Even weirder, when it tries to download the objects.inv from python.org it just hangs. If I manually cd to the build directory for gappy and run cd src/docs; make html the download works no problem. But when running ./sage -i gappy it hangs at:

[gappy-0.1.0a3] loading intersphinx inventory from https://docs.python.org/3/objects.inv...

Are we actually doing something to block the internet connection when installing packages?

comment:8

It eventually fails with

[gappy-0.1.0a3] WARNING: failed to reach any of the inventories with the following issues:
[gappy-0.1.0a3] intersphinx inventory 'https://docs.python.org/3/objects.inv' not fetchable due to <class 'requests.exceptions.ProxyError'>: HTTPSConnectionPool(host='docs.python.org', port=443): Max retries exceeded with url: /3/objects.inv (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f8dcb536518>: Failed to establish a new connection: [Errno 110] Connection timed out',)))
comment:9

A simple workaround suggested here is to write a wrapper around the original conf.py to override the intersphinx_mapping setting. I'm trying to get that to work now.

Branch pushed to git repo; I updated commit sha1. New commits:

5c9a06aSince this code was moved into a function it should use globals() not
092fa4bInstall the gappy docs if SAGE_SPKG_INSTALL_DOCS=yes

Changed commit from 943b009 to 092fa4b

comment:11

Yep, this approach gets the job done.

In case we also have a local copy of the Python docs installed, this can also change the intersphinx mapping to generate links to the local docs instead of the online ones, if desired. Same approach could be used in other packages (pplpy, etc.)

Next step: add the intersphinx mapping to the Sage docs, again generating links to the local gappy docs if available, and otherwise to the online docs.

Changed commit from 092fa4b to a87cfbe

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

86e4b1bDeprecate sage.libs.gap.element and convert Sage to use
2be416aSince this code was moved into a function it should use globals() not
45eeb8fInstall the gappy docs if SAGE_SPKG_INSTALL_DOCS=yes
a87cfbeVarious documentation improvements:
comment:13

This is nearly ready to go from my point of view, except that when I run the full test suite I get a weird segfault from the Singular interface around here:

sage: a = singular(1) ## line 502 ##
sage: _ = singular._expect.sendline('1+')  # unfinished input ## line 503 ##
sage: try:
    alarm(0.5)
    singular._expect_expr('>')  # interrupt this
except KeyboardInterrupt:
    pass ## line 504 ##
Control-C pressed. Interrupting Singular. Please wait a few seconds...
sage: 2*a ## line 513 ##

which seems to be emanating from sage.rings.integer

/home/embray/src/sagemath/sage/local/lib/python3.6/site-packages/cysignals/signals.cpython-36m-x86_64-linux-gnu.so(+0x6a6b)[0x7fc69ffd9a6b]
/home/embray/src/sagemath/sage/local/lib/python3.6/site-packages/cysignals/signals.cpython-36m-x86_64-linux-gnu.so(+0x6c48)[0x7fc69ffd9c48]
/home/embray/src/sagemath/sage/local/lib/python3.6/site-packages/cysignals/signals.cpython-36m-x86_64-linux-gnu.so(+0x9103)[0x7fc69ffdc103]
/lib/x86_64-linux-gnu/libc.so.6(+0x3efd0)[0x7fc6a6542fd0]
/home/embray/src/sagemath/sage/local/lib/python3.6/site-packages/cysignals/signals.cpython-36m-x86_64-linux-gnu.so(+0x9deb)[0x7fc69ffdcdeb]
/home/embray/src/sagemath/sage/local/lib/python3.6/site-packages/sage/rings/integer.cpython-36m-x86_64-linux-gnu.so(+0x16ab6)[0x7fc68bbd0ab6]
...

I'm not sure what this would possibly have to do with the libgap interface, but it's not outside the realm of possibility.

When I re-run the failing test by itself the segfault does not occur :/

comment:14

Should #31297 be listed as a dependency?

comment:15

Replying to @mkoeppe:

Should #31297 be listed as a dependency?

I don't think so. This supersedes #31297. Unless anyone has strong feelings that the wrapper approach is better, I think at some point that ticket should be closed in favor of this one.

comment:16

Thanks for the clarification. I have set the milestone of #31297 accordingly.

comment:17

I fought to make initialization of PermutationGroupElement much faster. The following bit reverts it completely when initializing from libgap permutations (using .ListPerm() go through the slow Python iteration protocol).

@@ -581,31 +576,11 @@ cdef class PermutationGroupElement(MultiplicativeGroupElement):
             ...
             ValueError: invalid data to initialize a permutation
         """
-        cdef UInt2* p2
-        cdef UInt4* p4
-        cdef int i
-        cdef UInt d
-
-        if TNUM_OBJ(p.value) == T_PERM2:
-            d = DEG_PERM2(p.value)
-            if d > self.n:
-                d = self.n
-            else:
-                for i in range(d, self.n):
-                    self.perm[i] = i
-            p2 = CONST_ADDR_PERM2(p.value)
-            for i in range(d):
-                self.perm[i] = p2[i]
-        elif TNUM_OBJ(p.value) == T_PERM4:
-            d = DEG_PERM4(p.value)
-            if d > self.n:
-                d = self.n
-            else:
-                for i in range(d, self.n):
-                    self.perm[i] = i
-            p4 = CONST_ADDR_PERM4(p.value)
-            for i in range(d):
-                self.perm[i] = p4[i]
+        if isinstance(p, GapPermutation):
+            try:
+                self._set_list_images(p.ListPerm(), False)
+            except AssertionError:
+                raise ValueError("invalid data to initialize a permutation")
         else:
             raise TypeError("not a gap permutation")

Is there any reason why not use the C API of GAP here?

comment:18

Replying to @videlec:

Is there any reason why not use the C API of GAP here?

Yes, the goal is to do away as much as possible with using undocumented/potentially unstable GAP internals. The GAP community has shown an interest of late of developing something more of a "public" API for using GAP as a library, and I and others from the Sage side have been working with them a lot to develop that.

Unfortunately, one of the "gaps" is public APIs for accessing permutation objects efficiently.

I'm glad you brought this up though because I was worried about exactly this change, and was not 100% sure how impactful it was. Is there an example case you can give me that might show whether or not there is a performance deficit?

I've made other performance improvements that might negate the change, but you are right that when using ListPerm it will, in the current code, have to iterate over the GAP List. Perhaps I can add a C method to GapPermutation to do this more efficiently.

comment:19

I'll add, if it makes an enormous difference, I can make an exception for now. It wouldn't be the only exception I've made in the current version of gappy for APIs that are not yet available in the libgap API. I'll propose some additions to libgap for accessing permutations, and then in gappy provide wrappers that provide the currently missing APIs.

comment:20

Some more info on the failing Singular test:

#0  0x00007fa76509e6d0 in __GI___waitpid () at /build/glibc-2ORdQG/glibc-2.27/posix/../sysdeps/unix/sysv/linux/waitpid.c:30
#1  0x00007fa75ea8fad0 in print_enhanced_backtrace () at /tmp/pip-req-build-b0moadh3/build/src/cysignals/implementation.c:563
#2  0x00007fa75ea8fc10 in sigdie () at /tmp/pip-req-build-b0moadh3/build/src/cysignals/implementation.c:589
#3  0x00007fa75ea91f31 in sigdie_for_sig () at /tmp/pip-req-build-b0moadh3/build/src/cysignals/implementation.c:164
#4  0x00007fa75ea91f00 in cysigs_signal_handler () at /tmp/pip-req-build-b0moadh3/build/src/cysignals/implementation.c:262
#5  0x00007fa7653c8170 in __restore_rt ()
#6  0x00007fa75ea92ac0 in __pyx_f_9cysignals_7signals_verify_exc_value () at /tmp/pip-req-build-b0moadh3/build/src/cysignals/signals.c:3736
#7  0x00007fa74a6989f9 in sig_occurred () at /home/embray/src/sagemath/sage/build/pkgs/sagelib/src/sage/modular/arithgroup/congroup.pyx:0
     1    """
     2    Cython helper functions for congruence subgroups
     3    
     4    This file contains optimized Cython implementations of a few functions related
#8  0x00007fa74a6989f0 in fast_tp_dealloc () at /home/embray/src/sagemath/sage/build/pkgs/sagelib/src/sage/rings/integer.pyx:7423
  7418        cdef mpz_ptr o_mpz = <mpz_ptr>((<Integer>o).value)
  7419    
  7420        # If we are recovering from an interrupt, throw the mpz_t away
  7421        # without recycling or freeing it because it might be in an
  7422        # inconsistent state (see Trac #24986).
> 7423        if sig_occurred() is NULL:
  7424            if integer_pool_count < integer_pool_size:
  7425                # Here we free any extra memory used by the mpz_t by
  7426                # setting it to a single limb.
  7427                if o_mpz._mp_alloc > 10:

The test in question is testing an alarm interrupt, so it seems maybe the interrupt is occurring before this sig_occurred() check in fast_tp_dealloc, which appears to be trying to do what it's supposed to do (don't recycle an mpz_t which might be in an inconsistent state). The rest of the stack trace above this isn't useful since it's using the system Python and I don't have the debugging symbols installed.

A couple weird things about this:

  • The segfault appears to be happening inside cysignals itself in verify_exc_value().

  • This happens consistently, in the same place, when I run the full test suite on this branch, but I can't seem to reproduce it in any other way, and I can't reproduce it on the current develop branch.

comment:21

Oh cool, just trying things at random, it seems I can reproduce it reliably with

./sage -t --file-iterations=2 src/sage/interfaces/singular.py

for some reason.

comment:22

I opened sagemath/cysignals#126 to track/discuss the cysignals issue. I wonder if anyone else can reproduce this. What's weird is that it seems to be pretty deterministic as far as when and how it happens, but hard to get it to happen in the first place, and also appears to be a rare corner case.

comment:23

I'm pretty sure that we have seen this failure in the singular interface quite consistently for a while now - thanks for isolating it - I don't think it has anything to do with the present ticket.

comment:24

Replying to @mkoeppe:

I'm pretty sure that we have seen this failure in the singular interface quite consistently for a while now - thanks for isolating it - I don't think it has anything to do with the present ticket.

Thanks for pointing that out. I'm glad I'm not the only one. Apparently this is tracked already in #30945.

Changed commit from a87cfbe to 24574e9

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

79a8f51Update gappy to v0.1.0a2 which adds MacOS and Cygwin fixes.
f2f1443Fixes needed for gappy v0.1.0a2 which did away with the libgap_soname
1e69f14Further rework of 45b175e43621795f890204a9cdfb30b524f961fa which
9efa4a3Update gappy to v0.1.3a3 with the requisite changes needed to adapt to
abe29b2A few minor test fixes:
28ea0faDon't assume that anything with a 'parent' attribute is fully
9ac9d9eDeprecate sage.libs.gap.element and convert Sage to use
9bd39d7Since this code was moved into a function it should use globals() not
de80b44Install the gappy docs if SAGE_SPKG_INSTALL_DOCS=yes
24574e9Various documentation improvements:

Changed commit from 24574e9 to 82a5f02

Branch pushed to git repo; I updated commit sha1. New commits:

82a5f02Replace a few more deprecated libgap.function_factory calls.
comment:27

GAP had a new release a week ago:

https://www.gap-system.org/Releases/4.11.1.html

Does gappy work with it?

comment:28

Replying to @dimpase:

GAP had a new release a week ago:

https://www.gap-system.org/Releases/4.11.1.html

Does gappy work with it?

Yes, it does. In fact the new release contains fixes that were motivated by gappy.

One thing I do need to add on the gappy side now is to either:

  1. Remove workarounds that were needed prior to this fixes and just make GAP 4.11.1 the minimum GAP version, or

  2. Do some version detection to decide what workarounds to include.

I'm going to try for the latter first.

comment:29

see #31498 for GAP upgrade

Work Issues: release gappy 0.1.0 final and update spkg version before merging

Changed work issues from release gappy 0.1.0 final and update spkg version before merging to release gappy 0.1.0 final and update spkg version before merging, test against p_groups_cohomology

comment:32

needs a rebase

comment:33

How far along is Sage 9.4 currently? I've been really wanting to get back to finishing up the first release of gappy and getting it into Sage, but my other projects have been slamming me the last several months :(

I hope by this summer I'll have time to do this, but it can wait for Sage 9.5 if need be...

comment:34

9.4 has not yet gotten its 1st beta.

comment:35

Setting a new milestone for this ticket based on a cursory review.

Changed commit from 82a5f02 to c5c84fb

comment:36

a straighforward rebase over 9.4.beta5 (hopefully correct)


Last 10 new commits:

4debcd5Fixes needed for gappy v0.1.0a2 which did away with the libgap_soname
d4e76c2Further rework of 45b175e43621795f890204a9cdfb30b524f961fa which
b544ad4Update gappy to v0.1.3a3 with the requisite changes needed to adapt to
c02e750A few minor test fixes:
112d154Don't assume that anything with a 'parent' attribute is fully
58a5880Deprecate sage.libs.gap.element and convert Sage to use
b90c248Since this code was moved into a function it should use globals() not
74dfdf8Install the gappy docs if SAGE_SPKG_INSTALL_DOCS=yes
f0e99fcVarious documentation improvements:
c5c84fbReplace a few more deprecated libgap.function_factory calls.
comment:37

it builds, but starting errors out:

cd "/home/scratch2/dimpase/sage/sage/src/doc" && sage-logger -p "make -j8 doc-html" /home/scratch2/dimpase/sage/sage/logs/dochtml.log
make[3]: Nothing to be done for 'all-sage'.
[dochtml] make[4]: warning: -jN forced in submake: disabling jobserver mode.
[dochtml] Traceback (most recent call last):
[dochtml]   File "/usr/lib64/python3.8/runpy.py", line 185, in _run_module_as_main
[dochtml]     mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
[dochtml]   File "/usr/lib64/python3.8/runpy.py", line 144, in _get_module_details
[dochtml]     return _get_module_details(pkg_main_name, error)
[dochtml]   File "/usr/lib64/python3.8/runpy.py", line 111, in _get_module_details
[dochtml]     __import__(pkg_name)
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage_docbuild/__init__.py", line 57, in <module>
[dochtml]     import sage.all
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/all.py", line 130, in <module>
[dochtml]     from sage.misc.all       import *         # takes a while
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/misc/all.py", line 90, in <module>
[dochtml]     from .functional import (additive_order,
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/misc/functional.py", line 25, in <module>
[dochtml]     from sage.rings.complex_double import CDF
[dochtml]   File "sage/rings/complex_double.pyx", line 101, in init sage.rings.complex_double (build/cythonized/sage/rings/complex_double.c:25297)
[dochtml]   File "sage/rings/complex_mpfr.pyx", line 200, in sage.rings.complex_mpfr.ComplexField (build/cythonized/sage/rings/complex_mpfr.c:5145)
[dochtml]   File "sage/rings/complex_mpfr.pyx", line 299, in sage.rings.complex_mpfr.ComplexField_class.__init__ (build/cythonized/sage/rings/complex_mpfr.c:5509)
[dochtml]   File "sage/rings/complex_mpfr.pyx", line 3365, in sage.rings.complex_mpfr.RRtoCC.__init__ (build/cythonized/sage/rings/complex_mpfr.c:28414)
[dochtml]   File "sage/categories/map.pyx", line 125, in sage.categories.map.Map.__init__ (build/cythonized/sage/categories/map.c:3579)
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/categories/homset.py", line 395, in Hom
[dochtml]     H = Hom(X, Y, category, check=False)
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/categories/homset.py", line 422, in Hom
[dochtml]     H = X._Hom_(Y, category)
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/categories/rings.py", line 422, in _Hom_
[dochtml]     from sage.rings.homset import RingHomset
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/rings/homset.py", line 18, in <module>
[dochtml]     from . import quotient_ring
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/rings/quotient_ring.py", line 113, in <module>
[dochtml]     import sage.rings.polynomial.multi_polynomial_ideal
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/rings/polynomial/multi_polynomial_ideal.py", line 235, in <module>
[dochtml]     from sage.interfaces.all import (singular as singular_default,
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/interfaces/all.py", line 10, in <module>
[dochtml]     from .gap import gap, gap_reset_workspace, set_gap_memory_pool_size, Gap
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/interfaces/gap.py", line 180, in <module>
[dochtml]     from .gap_workspace import gap_workspace_file, prepare_workspace_dir
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/interfaces/gap_workspace.py", line 19, in <module>
[dochtml]     from sage.env import DOT_SAGE, GAP_SO
[dochtml] ImportError: cannot import name 'GAP_SO' from 'sage.env' (/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/env.py)
[dochtml] make -j8 doc-inventory--
[dochtml] make[5]: warning: -jN forced in submake: disabling jobserver mode.
[dochtml] cd /home/scratch2/dimpase/sage/sage && ./sage --docbuild --no-pdf-links .o inventory
[dochtml] Traceback (most recent call last):
[dochtml]   File "/usr/lib64/python3.8/runpy.py", line 185, in _run_module_as_main
[dochtml]     mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
[dochtml]   File "/usr/lib64/python3.8/runpy.py", line 144, in _get_module_details
[dochtml]     return _get_module_details(pkg_main_name, error)
[dochtml]   File "/usr/lib64/python3.8/runpy.py", line 111, in _get_module_details
[dochtml]     __import__(pkg_name)
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage_docbuild/__init__.py", line 57, in <module>
[dochtml]     import sage.all
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/all.py", line 130, in <module>
[dochtml]     from sage.misc.all       import *         # takes a while
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/misc/all.py", line 90, in <module>
[dochtml]     from .functional import (additive_order,
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/misc/functional.py", line 25, in <module>
[dochtml]     from sage.rings.complex_double import CDF
[dochtml]   File "sage/rings/complex_double.pyx", line 101, in init sage.rings.complex_double (build/cythonized/sage/rings/complex_double.c:25297)
[dochtml]   File "sage/rings/complex_mpfr.pyx", line 200, in sage.rings.complex_mpfr.ComplexField (build/cythonized/sage/rings/complex_mpfr.c:5145)
[dochtml]   File "sage/rings/complex_mpfr.pyx", line 299, in sage.rings.complex_mpfr.ComplexField_class.__init__ (build/cythonized/sage/rings/complex_mpfr.c:5509)
[dochtml]   File "sage/rings/complex_mpfr.pyx", line 3365, in sage.rings.complex_mpfr.RRtoCC.__init__ (build/cythonized/sage/rings/complex_mpfr.c:28414)
[dochtml]   File "sage/categories/map.pyx", line 125, in sage.categories.map.Map.__init__ (build/cythonized/sage/categories/map.c:3579)
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/categories/homset.py", line 395, in Hom
[dochtml]     H = Hom(X, Y, category, check=False)
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/categories/homset.py", line 422, in Hom
[dochtml]     H = X._Hom_(Y, category)
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/categories/rings.py", line 422, in _Hom_
[dochtml]     from sage.rings.homset import RingHomset
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/rings/homset.py", line 18, in <module>
[dochtml]     from . import quotient_ring
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/rings/quotient_ring.py", line 113, in <module>
[dochtml]     import sage.rings.polynomial.multi_polynomial_ideal
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/rings/polynomial/multi_polynomial_ideal.py", line 235, in <module>
[dochtml]     from sage.interfaces.all import (singular as singular_default,
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/interfaces/all.py", line 10, in <module>
[dochtml]     from .gap import gap, gap_reset_workspace, set_gap_memory_pool_size, Gap
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/interfaces/gap.py", line 180, in <module>
[dochtml]     from .gap_workspace import gap_workspace_file, prepare_workspace_dir
[dochtml]   File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/interfaces/gap_workspace.py", line 19, in <module>
[dochtml]     from sage.env import DOT_SAGE, GAP_SO
[dochtml] ImportError: cannot import name 'GAP_SO' from 'sage.env' (/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/env.py)
[dochtml] make[5]: *** [Makefile:20: doc-inventory--.o] Error 1
[dochtml] make[4]: *** [Makefile:35: doc-inventory-reference] Error 2
make[3]: *** [Makefile:2359: doc-html] Error 2
make[2]: *** [Makefile:2245: all-start] Error 2
make[2]: Leaving directory '/home/scratch2/dimpase/sage/sage/build/make'

is it something trivial?

Reviewer: Dima Pasechnik

comment:39

well:

src/sage/interfaces/gap_workspace.py:from sage.env import DOT_SAGE, GAP_SO

and is not touched by the branch...

comment:40

That use of GAP_SO, which causes the crash, is pretty recent:

$ git show a801e6d85bd
commit a801e6d85bd420b60ea75b1671856eb43ac6b18b
Author: Matthias Koeppe <mkoeppe@math.ucdavis.edu>
Date:   Wed Jun 23 14:50:08 2021 -0700

    src/sage/interfaces/gap_workspace.py: Use hash of GAP_SO to disambiguate the workspace file, not SAGE_LOCAL

diff --git a/src/sage/interfaces/gap_workspace.py b/src/sage/interfaces/gap_workspace.py
index 953dc85116..33a87dd507 100644
--- a/src/sage/interfaces/gap_workspace.py
+++ b/src/sage/interfaces/gap_workspace.py
@@ -16,7 +16,7 @@ Support for (lib)GAP workspace files
 import os
 import time
 import hashlib
-from sage.env import DOT_SAGE, SAGE_LOCAL
+from sage.env import DOT_SAGE, GAP_SO
 
 
 def gap_workspace_file(system="gap", name="workspace", dir=None):
@@ -59,7 +59,10 @@ def gap_workspace_file(system="gap", name="workspace", dir=None):
     if dir is None:
         dir = os.path.join(DOT_SAGE, 'gap')
 
-    h = hashlib.sha1(SAGE_LOCAL.encode('utf-8')).hexdigest()
+    if GAP_SO:
+        h = hashlib.sha1(GAP_SO.encode('utf-8')).hexdigest()
+    else:
+        h = 'unknown'
     return os.path.join(dir, '%s-%s-%s' % (system, name, h))
 
 

Reverting it allows Sage to start. Matthias, how should this be properly dealt with?

comment:41

Apart from this, GAP workspace management is broken:

**********************************************************************
File "src/sage/libs/gap/libgap.pyx", line 24, in sage.libs.gap.libgap
Failed example:
    a = libgap(10)
Exception raised:
    Traceback (most recent call last):
      File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/doctest/forker.py", line 718, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/scratch2/dimpase/sage/sage/local/lib64/python3.8/site-packages/sage/doctest/forker.py", line 1137, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.libs.gap.libgap[0]>", line 1, in <module>
        a = libgap(Integer(10))
      File "sage/misc/lazy_import.pyx", line 362, in sage.misc.lazy_import.LazyImport.__call__ (build/cythonized/sage/misc/lazy_import.c:4036)
        return self.get_object()(*args, **kwds)
      File "gappy/core.pyx", line 691, in gappy.core.Gap.__call__
        self.initialize()
      File "sage/libs/gap/libgap.pyx", line 274, in sage.libs.gap.libgap.SageGap.initialize (build/cythonized/sage/libs/gap/libgap.c:2786)
        initializing = Gap.initialize(self)
      File "gappy/core.pyx", line 603, in gappy.core.Gap.initialize
        self._init_kwargs.update(initialize(
      File "gappy/core.pyx", line 253, in gappy.core.initialize
        with open(workspace, 'rb'):
    FileNotFoundError: [Errno 2] No such file or directory: '/home/scratch/dimpase/.sage/gap/libgap-workspace-f048f6ddda94b972db915da793e521b0a11cd05f'
...
comment:42

Replying to @dimpase:

That use of GAP_SO, which causes the crash, is pretty recent:

$ git show a801e6d85bd
commit a801e6d85bd420b60ea75b1671856eb43ac6b18b
Author: Matthias Koeppe <mkoeppe@math.ucdavis.edu>
Date:   Wed Jun 23 14:50:08 2021 -0700

    src/sage/interfaces/gap_workspace.py: Use hash of GAP_SO to disambiguate the workspace file, not SAGE_LOCAL

diff --git a/src/sage/interfaces/gap_workspace.py b/src/sage/interfaces/gap_workspace.py
index 953dc85116..33a87dd507 100644
--- a/src/sage/interfaces/gap_workspace.py
+++ b/src/sage/interfaces/gap_workspace.py
@@ -16,7 +16,7 @@ Support for (lib)GAP workspace files
 import os
 import time
 import hashlib
-from sage.env import DOT_SAGE, SAGE_LOCAL
+from sage.env import DOT_SAGE, GAP_SO
 
 
 def gap_workspace_file(system="gap", name="workspace", dir=None):
@@ -59,7 +59,10 @@ def gap_workspace_file(system="gap", name="workspace", dir=None):
     if dir is None:
         dir = os.path.join(DOT_SAGE, 'gap')
 
-    h = hashlib.sha1(SAGE_LOCAL.encode('utf-8')).hexdigest()
+    if GAP_SO:
+        h = hashlib.sha1(GAP_SO.encode('utf-8')).hexdigest()
+    else:
+        h = 'unknown'
     return os.path.join(dir, '%s-%s-%s' % (system, name, h))
 
 

Reverting it allows Sage to start. Matthias, how should this be properly dealt with?

I don't have much of an idea about GAP workspaces. Before my change, a unique identifier was obtained keyed to SAGE_LOCAL. I figured this was supposed to somehow capture version dependence. So keying it to GAP_SO was my solution to remove the use of SAGE_LOCAL. I suppose any kind of version info of gappy would also do the job.

Changed commit from c5c84fb to 049ec50

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

260c19fFixes needed for gappy v0.1.0a2 which did away with the libgap_soname
78a7b7bFurther rework of 45b175e43621795f890204a9cdfb30b524f961fa which
a2463d8Update gappy to v0.1.3a3 with the requisite changes needed to adapt to
58560a9A few minor test fixes:
b7c0a31Don't assume that anything with a 'parent' attribute is fully
fb86c3cDeprecate sage.libs.gap.element and convert Sage to use
111760cSince this code was moved into a function it should use globals() not
343b898Install the gappy docs if SAGE_SPKG_INSTALL_DOCS=yes
42a115cVarious documentation improvements:
049ec50Replace a few more deprecated libgap.function_factory calls.
comment:44

OK, this branch at least builds and it's possible to start Sage, and build docs. Some doctests in sage.geometry.polyhedron.base fail, however:

...
File "src/sage/geometry/polyhedron/base.py", line 10469, in sage.geometry.polyhedron.base.Polyhedron_base.restricted_automorphism_group
Failed example:
    P.restricted_automorphism_group(output="matrixlist")[0].is_immutable()
Exception raised:
    Traceback (most recent call last):
      File "sage/misc/cachefunc.pyx", line 1943, in sage.misc.cachefunc.CachedMethodCaller.__call__ (build/cythonized/sage/misc/cachefunc.c:10347)
        return cache[k]
    KeyError: (('matrixlist',), ())

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 718, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1137, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.geometry.polyhedron.base.Polyhedron_base.restricted_automorphism_group[43]>", line 1, in <module>
        P.restricted_automorphism_group(output="matrixlist")[Integer(0)].is_immutable()
      File "sage/misc/cachefunc.pyx", line 1948, in sage.misc.cachefunc.CachedMethodCaller.__call__ (build/cythonized/sage/misc/cachefunc.c:10483)
        w = self._instance_call(*args, **kwds)
      File "sage/misc/cachefunc.pyx", line 1824, in sage.misc.cachefunc.CachedMethodCaller._instance_call (build/cythonized/sage/misc/cachefunc.c:9949)
        return self.f(self._instance, *args, **kwds)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/geometry/polyhedron/base.py", line 10563, in restricted_automorphism_group
        permgroup = G.automorphism_group(edge_labels=True)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/graphs/generic_graph.py", line 22260, in automorphism_group
        output.append(PermutationGroup(gens=gens, domain=int_to_vertex.values()))
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/groups/perm_gps/permgroup.py", line 352, in PermutationGroup
        return PermutationGroup_generic(gens=gens, gap_group=gap_group, domain=domain,
    NameError: name 'gap_group' is not defined
**********************************************************************
4 items had failures:
   9 of  15 in sage.geometry.polyhedron.base.Polyhedron_base.combinatorial_automorphism_group
  28 of  45 in sage.geometry.polyhedron.base.Polyhedron_base.restricted_automorphism_group
   1 of  15 in sage.geometry.polyhedron.base.Polyhedron_base.vertex_facet_graph
   1 of  10 in sage.geometry.polyhedron.base.Polyhedron_base.vertex_graph
    [1755 tests, 39 failures, 121.34 s]

Changed commit from 049ec50 to 6c99fbf

Branch pushed to git repo; I updated commit sha1. New commits:

6c99fbfrestore preparing PermutationGroup_generic() args

Changed commit from 6c99fbf to 69feb67

Branch pushed to git repo; I updated commit sha1. New commits:

69feb67restore computing GAP_SO
comment:47

libgap workspace management is still broken, as can be seen by removing ~/.sage/gap/ and
running e.g. sage: libgap(1) at Sage prompt.

sage: libgap(1)
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-1-037524c8c625> in <module>
----> 1 libgap(Integer(1))

/home/scratch2/dimpase/sage/sage/local/lib64/python3.9/site-packages/sage/misc/lazy_import.pyx in sage.misc.lazy_import.LazyImport.__call__ (build/cythonized/sage/misc/lazy_import.c:4052)()
    360             True
    361         """
--> 362         return self.get_object()(*args, **kwds)
    363
    364     def __repr__(self):

/home/scratch2/dimpase/sage/sage/local/lib64/python3.9/site-packages/gappy/core.pyx in gappy.core.Gap.__call__()
    689
    690         """
--> 691         self.initialize()
    692         if isinstance(x, GapObj):
    693             return x

/home/scratch2/dimpase/sage/sage/local/lib64/python3.9/site-packages/sage/libs/gap/libgap.pyx in sage.libs.gap.libgap.SageGap.initialize (build/cythonized/sage/libs/gap/libgap.c:2801)()
    272         """
    273
--> 274         initializing = Gap.initialize(self)
    275
    276         if initializing:

/home/scratch2/dimpase/sage/sage/local/lib64/python3.9/site-packages/gappy/core.pyx in gappy.core.Gap.initialize()
    601             return False
    602
--> 603         self._init_kwargs.update(initialize(
    604             gap_root=self._init_kwargs['gap_root'],
    605             gaprc=self._init_kwargs['gaprc'],

/home/scratch2/dimpase/sage/sage/local/lib64/python3.9/site-packages/gappy/core.pyx in gappy.core.initialize()
    251         workspace = os.path.normpath(workspace)
    252
--> 253         with open(workspace, 'rb'):
    254             pass
    255

FileNotFoundError: [Errno 2] No such file or directory: '/home/scratch/dimpase/.sage/gap/libgap-workspace-c357604137f37c963401793ca1ab74cce6c00efd'
comment:48

I see how the workspace issue can be fixed, but it might need a change in gappy - I opened embray/gappy#16 to discuss.

Upstream: Reported upstream. No feedback yet.

Changed commit from 69feb67 to 1b5adee

Branch pushed to git repo; I updated commit sha1. New commits:

1b5adeegappy PR#17 patch; uses it to handle workspaces

Changed reviewer from Dima Pasechnik to Dima Pasechnik, ...

comment:50

we still need to carefully check that GAP workspace rotation is not broken, but hopefully not.

comment:51

a couple of test failures I can't quite make sense of. The 1st one has something to do with
cachefunc, categories, etc...

File "src/sage/groups/perm_gps/permgroup_morphism.py", line 119, in sage.groups.perm_gps.permgroup_morphism.PermutationGroupMorphism.image
Failed example:
    h = D4.isomorphism_to(G)
Exception raised:
    Traceback (most recent call last):
      File "sage/misc/cachefunc.pyx", line 1943, in sage.misc.cachefunc.CachedMethodCaller.__call__ (build/cythonized/sage/misc/cachefunc.c:10347)
        return cache[k]
    KeyError: ((<SageGap(gap_root='/mnt/opt/Sage/sage-dev/local/share/gap')>,), ())

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "sage/structure/category_object.pyx", line 839, in sage.structure.category_object.CategoryObject.getattr_from_category (build/cythonized/sage/structure/category_object.c:7089)
        return self.__cached_methods[name]
    KeyError: '_pari_nf'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/rings/number_field/number_field.py", line 4348, in pari_nf
        return self._pari_nf
      File "sage/structure/category_object.pyx", line 833, in sage.structure.category_object.CategoryObject.__getattr__ (build/cythonized/sage/structure/category_object.c:7008)
        return self.getattr_from_category(name)
      File "sage/structure/category_object.pyx", line 848, in sage.structure.category_object.CategoryObject.getattr_from_category (build/cythonized/sage/structure/category_object.c:7174)
        attr = getattr_from_other_class(self, cls, name)
      File "sage/cpython/getattr.pyx", line 367, in sage.cpython.getattr.getattr_from_other_class (build/cythonized/sage/cpython/getattr.c:2566)
        raise AttributeError(dummy_error_message)
    AttributeError: 'NumberField_absolute_with_category' object has no attribute '_cache___pari_absolute_structure'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 718, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1137, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.groups.perm_gps.permgroup_morphism.PermutationGroupMorphism.image[17]>", line 1, in <module>
        h = D4.isomorphism_to(G)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/groups/perm_gps/permgroup.py", line 4076, in isomorphism_to
        iso = self._libgap_().IsomorphismGroups(right)
      File "gappy/gapobj.pyx", line 2773, in gappy.gapobj.GapMethodProxy.__call__
        return self.func.__call__(self.self, *args)
      File "gappy/gapobj.pyx", line 2489, in gappy.gapobj.GapFunction.__call__
        sig_on()
    SystemError: calling remove_from_pari_stack() inside sig_on()

The 2nd one is even less explicit, with a mysterious
SyntaxWarning: 'tuple' object is not callable; perhaps you missed a comma?
coming from somewhere I don't see

File "src/sage/groups/perm_gps/permgroup.py", line 614, in sage.groups.perm_gps.permgroup.PermutationGroup_generic.gap
Failed example:
    gap(P8) == P8.gap()
Expected:
    True
Got:
    doctest:warning
      File "/mnt/opt/Sage/sage-dev/src/bin/sage-runtests", line 144, in <module>
        err = DC.run()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/control.py", line 1207, in run
        self.run_doctests()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/control.py", line 909, in run_doctests
        self.dispatcher.dispatch()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2044, in dispatch
        self.parallel_dispatch()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1939, in parallel_dispatch
        w.start()  # This might take some time
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2211, in start
        super(DocTestWorker, self).start()
      File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start
        self._popen = self._Popen(self)
      File "/usr/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/usr/lib/python3.8/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
        self._launch(process_obj)
      File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch
        code = process_obj._bootstrap(parent_sentinel=child_r)
      File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
        self.run()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2183, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2512, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2559, in _run
        result = runner.run(test)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 910, in run
        return self._run(test, compileflags, out)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 718, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1137, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.groups.perm_gps.permgroup.PermutationGroup_generic.gap[2]>", line 1, in <module>
        gap(P8) == P8.gap()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/interfaces/interface.py", line 1082, in _sage_
        return sage.misc.sage_eval.sage_eval(string)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/misc/sage_eval.py", line 201, in sage_eval
        return eval(source, sage.all.__dict__, locals)
      File "/usr/lib/python3.8/warnings.py", line 109, in _showwarnmsg
        sw(msg.message, msg.category, msg.filename, msg.lineno,
    :
    SyntaxWarning: 'tuple' object is not callable; perhaps you missed a comma?
    doctest:warning
      File "/mnt/opt/Sage/sage-dev/src/bin/sage-runtests", line 144, in <module>
        err = DC.run()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/control.py", line 1207, in run
        self.run_doctests()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/control.py", line 909, in run_doctests
        self.dispatcher.dispatch()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2044, in dispatch
        self.parallel_dispatch()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1939, in parallel_dispatch
        w.start()  # This might take some time
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2211, in start
        super(DocTestWorker, self).start()
      File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start
        self._popen = self._Popen(self)
      File "/usr/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/usr/lib/python3.8/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
        self._launch(process_obj)
      File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch
        code = process_obj._bootstrap(parent_sentinel=child_r)
      File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
        self.run()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2183, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2512, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 2559, in _run
        result = runner.run(test)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 910, in run
        return self._run(test, compileflags, out)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 718, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/doctest/forker.py", line 1137, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.groups.perm_gps.permgroup.PermutationGroup_generic.gap[2]>", line 1, in <module>
        gap(P8) == P8.gap()
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/interfaces/interface.py", line 1082, in _sage_
        return sage.misc.sage_eval.sage_eval(string)
      File "/mnt/opt/Sage/sage-dev/local/lib/python3.8/site-packages/sage/misc/sage_eval.py", line 201, in sage_eval
        return eval(source, sage.all.__dict__, locals)
      File "/usr/lib/python3.8/warnings.py", line 109, in _showwarnmsg
        sw(msg.message, msg.category, msg.filename, msg.lineno,
    :
    SyntaxWarning: 'tuple' object is not callable; perhaps you missed a comma?
    True

Branch pushed to git repo; I updated commit sha1. New commits:

3f1d051make invocation of _libgap_() explicit

Changed commit from 1b5adee to 3f1d051

comment:53

I still wouldn't move forward on this too much until I have a first release of gappy out. I haven't had time to work on it in several months, so it's probably fallen a bit behind the last GAP releases (and can probably drop some workarounds for older GAP versions).

Other than that there isn't too much holding up making a gappy release--I think, as you found, some of the workspace management stuff, and some of the global namespace stuff from Sage's libgap needs to be worked into it better.

comment:54

Stalled in needs_review or needs_info; likely won't make it into Sage 9.5.

comment:55

See also: #33072.

slel commented

Description changed:

--- 
+++ 
@@ -1,5 +1,11 @@
+From [the gappy README](https://github.com/embray/gappy#gappy--a-python-interface-to-gap):
+
+> gappy provides a Python interface to the GAP
+> computer algebra system by linking to its
+> library interface.
+
 This is a follow-up to #31297, particularly inspired by [#31297 comment:17](https://github.com/sagemath/sage/issues/31297#comment:17).  It is a more disruptive change, in that instead of providing wrappers around gappy that are Sage Parents and Elements, it follows the example of cypari2 and just uses gappy more-or-less directly without any wrappers.
 
 It remains completely agnostic to the coercion system, though I am not completely happy with this state of affairs.  In particular you can see I had to add a special case to `Polynomial.__call__` for handling evaluating polynomials on `GapObj`s, a case that used to work fine, but now needs a special case since other Sage types cannot be coerced to `GapObj`s.
 
-Also had to add special cases for instantiating Integers and Rationals from GapObjs, but on the plus side this is now a bit faster.
+Also had to add special cases for instantiating `Integer`s and `Rational`s from `GapObj`s, but on the plus side this is now a bit faster.
comment:57

Moving to "needs work," as there's nothing to review at the moment (we're waiting for an upstream gappy release).

comment:58

Upstream changed jobs recently, not sure if an update will be soon...