Replacement of sage/libs/ppl.pyx by pplpy
vinklein opened this issue · 255 comments
The Sage wrappers around ppl are now superseded by the standalone pplpy (Python/Cython compatible). This ticket proposes to replace src/sage/libs/ppl.pyx extension with pplpy. For this, we
- create a new standard package pplpy
- upgrade gmpy2 from optional to standard
- replace the Sage code calling sage/libs/ppl.pyx to call pplpy
- make appropriate deprecation for
sage.libs.ppl
Tarball for pplpy: pplpy-0.8.4.tar.gz
Upstream: Reported upstream. Developers acknowledge bug.
CC: @videlec @kiwifb @embray @slel
Component: packages: standard
Keywords: thursdaysbdx, ppl, pplpy, upgrade
Author: Vincent Klein
Branch: 829317e
Reviewer: Dima Pasechnik, François Bissey, Vincent Delecroix, Jeroen Demeyer, Erik Bray
Issue created by migration from https://trac.sagemath.org/ticket/23024
Description changed:
---
+++
@@ -1,4 +1,4 @@
-Remove ppl.pyx extension and replace with pplpy[https://github.com/videlec/pplpy/](https://github.com/videlec/pplpy/) package.
+The Sage wrappers around [ppl](http://bugseng.com/products/ppl/) is now superseded by the stand alone [pplpy](https://github.com/videlec/pplpy/) (Python/Cython compatible). This ticket proposes to replace `sage/src/libs/ppl.pyx` extension with pplpy.
Some steps :
- list any ticket proposing to extend ppl functionalities in Sage (if yes, open issues on github with cross links)Description changed:
---
+++
@@ -1,4 +1,4 @@
-The Sage wrappers around [ppl](http://bugseng.com/products/ppl/) is now superseded by the stand alone [pplpy](https://github.com/videlec/pplpy/) (Python/Cython compatible). This ticket proposes to replace `sage/src/libs/ppl.pyx` extension with pplpy.
+The Sage wrappers around [ppl](http://bugseng.com/products/ppl/) is now superseded by the stand alone [pplpy](https://github.com/videlec/pplpy/) (!Python/!Cython compatible). This ticket proposes to replace `sage/src/libs/ppl.pyx` extension with pplpy.
Some steps :
- list any ticket proposing to extend ppl functionalities in Sage (if yes, open issues on github with cross links)Branch pushed to git repo; I updated commit sha1. New commits:
f8bf274 | Updating checksum for pplpy pkg |
Commit: f8bf274
For testing purpose you can use the following tarball of pplpy (outdated, use the one in the ticket's description)
Salut Vincent,
Your branch appears in red which means that there are merge issues (possibly trivial). Would be better to rebase it on the latest beta.
- Pickling should be fixed
- sage: TestSuite(Asso).run()
+ sage: TestSuite(Asso).run(skip='_test_pickling')It works with pplpy used from Python
>>> x = ppl.Variable(0)
>>> y = ppl.Variable(1)
>>> z = ppl.Variable(2)
>>> gs.insert(ppl.point(x+y+2*z))
>>> gs.insert(ppl.point(x-y))
>>> gs.insert(ppl.point(z))
>>> P = ppl.C_Polyhedron(gs)
>>> import pickle
>>> pickle.loads(pickle.dumps(P)) == P
True
- I don't see the point of the function
mpz_to_sage_integer. You can just do[Integer(x) for x in my_mpz_list]. For example
v_coords = (1,) + tuple(mpz_to_sage_integer(v.coefficients()))
can be replaced by
v_coords = (1,) + tuple(Integer(c) for c in v.coefficients())
Moreover, the name is confusing since the fuction converts a list of mpz integers.
- Some possible simplifications
-
In
ppl_backend.pyx:
Integer(g.coefficient(Variable(variable))) / Integer(g.divisor())->
Rational(g.coefficient(Variable(variable)) / g.divisor()) -
In
ppl_lattice_polytope.py:assert Integer(v.divisor()).is_one()->assert v.divisor() == 1
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
f19f81e | Integer and rational constructors support gmpy2 params |
cd8ef5d | remove OptionalExtension sage.libs.gmpy2 |
8ced5d3 | remove get_gmpy2_path() |
674fe62 | Add few doctests |
2ed4f95 | Use of MPZ_Check and MPQ_Check functions from gmpy2 C-API |
b4f0c26 | Add pplpy as a standard package |
257c330 | Replace sage.libs.ppl with pplpy package |
636051f | Remove sage.libs.ppl extension |
f52b3d4 | Modify associahedron doctest |
dffa134 | Updating checksum for pplpy pkg |
Replying to @videlec:
- Some possible simplifications
In
ppl_backend.pyx:
Integer(g.coefficient(Variable(variable))) / Integer(g.divisor())->
Rational(g.coefficient(Variable(variable)) / g.divisor())In
ppl_lattice_polytope.py:assert Integer(v.divisor()).is_one()->assert v.divisor() == 1
It will be nice, but coercion for gmpy2 types is required for doing this (#23052).
Branch pushed to git repo; I updated commit sha1. New commits:
b3563eb | remove mpz_to_sage_integer function |
Replying to @vinklein:
Replying to @videlec:
- Some possible simplifications
In
ppl_backend.pyx:
Integer(g.coefficient(Variable(variable))) / Integer(g.divisor())->
Rational(g.coefficient(Variable(variable)) / g.divisor())In
ppl_lattice_polytope.py:assert Integer(v.divisor()).is_one()->assert v.divisor() == 1It will be nice, but coercion for gmpy2 types is required for doing this (#23052).
Where? The first involves a direct call Rational(mpq_type) and the second comparison between mpz and Python integer. Which operation is mixing gmpy2 types with Sage types?
Replying to @videlec:
Where? The first involves a direct call
Rational(mpq_type)and the second comparison betweenmpzand Python integer. Which operation is mixing gmpy2 types with Sage types?
You're right, tests failures in ppl_backend.pyx with the suggested code are not coercion related. Forget my previous comment.
As pplpy has evolved to fix pickling with cPickle download pplpy again to update your local package.
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
08c7a72 | Add path of gmpy2 in include_dirs |
359a0ff | update checksum and package version |
0d4fd2e | Integer and rational constructors support gmpy2 params |
7c66a7c | remove OptionalExtension sage.libs.gmpy2 |
d143dde | remove get_gmpy2_path() |
2c56a3f | Add few doctests |
e5264fd | Use of MPZ_Check and MPQ_Check functions from gmpy2 C-API |
504ab68 | Change way of creating a Rational from a mpz |
072a15c | add __mpz__() to Rational |
858c2e5 | trac 23024 update pplpy checksum |
As pplpy has evolved to fix pickling with cPickle download pplpy again to update your local package.
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
1c1b971 | Replace sage.libs.ppl with pplpy package |
2a1798e | Remove sage.libs.ppl extension |
d1b0b2f | Modify associahedron doctest |
db21e1d | Updating checksum for pplpy pkg |
eb215c3 | remove mpz_to_sage_integer function |
53f30f3 | assert simplification ppl_lattice_polytope.py |
f9a6415 | remove skip=('_test_pickling') option for pplpy polyhedron |
924984c | remove sage/libs/ppl from doctree |
ce863ff | update checksum, snapshot version update in setup.py |
7a5812a | Add path of gmpy2 in include_dirs |
Description changed:
---
+++
@@ -1,6 +1,7 @@
The Sage wrappers around [ppl](http://bugseng.com/products/ppl/) is now superseded by the stand alone [pplpy](https://github.com/videlec/pplpy/) (!Python/!Cython compatible). This ticket proposes to replace `sage/src/libs/ppl.pyx` extension with pplpy.
Some steps :
+- pplpy and gmpy2 must be standard packages.
- list any ticket proposing to extend ppl functionalities in Sage (if yes, open issues on github with cross links)
- see if anybody changed sage/libs/ppl.pyx since the fork (if yes, list the commit and port them into pplpy)
- prepare a package for pplpy as documented in the developer guideBranch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
760bbdf | Modify associahedron doctest |
6d3345e | Updating checksum for pplpy pkg |
85cb9fc | remove mpz_to_sage_integer function |
70177c8 | assert simplification ppl_lattice_polytope.py |
b0858a2 | remove skip=('_test_pickling') option for pplpy polyhedron |
94c7817 | remove sage/libs/ppl from doctree |
67c8c2e | update checksum, snapshot version update in setup.py |
50d62ff | Add path of gmpy2 in include_dirs |
375fda6 | Trac #23024 Fix checksum (pplpy), spkg-install and spkg-check |
dc37f38 | Trac #23024 Add new mpz gmpy2 to sage Integer conversion |
Rebase on 22928 (with 22928 based on 8.2.beta0)
You need to provide a link to the pplpy tarball.
And proper deprecation has to be done for sage.libs.ppl so that a user doing
sage: from sage.libs.ppl import whatever
should not get an ImportError but a warning. See http://doc.sagemath.org/html/en/reference/misc/sage/misc/superseded.htm
Description changed:
---
+++
@@ -1,8 +1,6 @@
-The Sage wrappers around [ppl](http://bugseng.com/products/ppl/) is now superseded by the stand alone [pplpy](https://github.com/videlec/pplpy/) (!Python/!Cython compatible). This ticket proposes to replace `sage/src/libs/ppl.pyx` extension with pplpy.
+The Sage wrappers around [ppl](http://bugseng.com/products/ppl/) are now superseded by the standalone [pplpy](https://github.com/videlec/pplpy/) (Python/Cython compatible). This ticket proposes to replace `sage/src/libs/ppl.pyx` extension with pplpy. It
-Some steps :
-- pplpy and gmpy2 must be standard packages.
-- list any ticket proposing to extend ppl functionalities in Sage (if yes, open issues on github with cross links)
-- see if anybody changed sage/libs/ppl.pyx since the fork (if yes, list the commit and port them into pplpy)
-- prepare a package for pplpy as documented in the developer guide
+- creates a new standard package pplpy
+- upgrade gmpy2 from optional to standard
- replace the Sage code calling sage/libs/ppl.pyx to call pplpy
+- make appropriate deprecation for `sage.libs.ppl`Description changed:
---
+++
@@ -4,3 +4,5 @@
- upgrade gmpy2 from optional to standard
- replace the Sage code calling sage/libs/ppl.pyx to call pplpy
- make appropriate deprecation for `sage.libs.ppl`
+
+pplpy' Tarball : [pplpy-0.7.tar.gz](https://pypi.python.org/packages/ca/d3/1629e95997068c129e3a5235fd32a39e4fc39faca892cf9a2da11f842fb9/pplpy-0.7.tar.gz#md5=89dd6a1e75e06e287adae2ea4dbc143b)And by proper deprecation do you mean keeping sage.libs.ppl fully functioning with his doctests ? Or do we prefer to shutdown sage.libs.ppl functionalities (Functions don't do their old work and just return a warning) ?
Replying to @vinklein:
And by proper deprecation do you mean keeping sage.libs.ppl fully functioning with his doctests ? Or do we prefer to shutdown sage.libs.ppl functionalities (Functions don't do their old work and just return a warning) ?
The aim is not to keep sage.libs.ppl but to just to have some indirection in the import so that you do not break user's code. As sage.libs.ppl will not here anymore I do not expect the full doctesuite to pass as it used to be. But the following kind of things should work (and be doctested)
sage: from sage.libs.ppl import Variable, Constraint_System, C_Polyhedron
UserWarning: proper deprecation message
sage: x = Variable(0)
sage: y = Variable(1)
sage: cs = Constraint_System()
sage: cs.insert(x >= 0)
sage: cs.insert(x <= 3)
sage: cs.insert(y >= 0)
sage: cs.insert(y <= 3)
sage: poly_from_constraints = C_Polyhedron(cs)
And the various objects constructed above are of course from pplpy.
You can have a look at other places in the sage code and look for "deprecation" and "lazy_import".
Well, i spoke with Sebastien L. and if the goal is to let one year for developers to adapt their old code, doing an indirection is not enough. Mainly because pplpy use gmpy2 (some functions return and calls are differents).
Therefore it seems like we should keep sage.libs.ppl old code with added warnings.
Just a small request can you put upstream's address in SPKG.txt? Of course you are available on pypi but it is not the same as being able to find where to fill bugs and what the current issues are straight away.
Branch pushed to git repo; I updated commit sha1. New commits:
7d19154 | Trac #23024 SPKG.txt add upstream contact and licence. |
Done, thanks for noticing that.
Branch pushed to git repo; I updated commit sha1. New commits:
01403bd | Trac #23024 Deprecate sage.libs.ppl |
Deprecation of sage.libs.ppl added. As the whole module is deprecated i choose to send deprecation warning at import time.
Did you run the testsuite!?
sage -t src/sage/geometry/lattice_polytope.py
**********************************************************************
File "src/sage/geometry/lattice_polytope.py", line 1608, in sage.geometry.lattice_polytope.LatticePolytopeClass.boundary_point_indices
Failed example:
face.points()
Exception raised:
Traceback (most recent call last):
File "/home/vdelecro/sage_patchbot/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 517, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/home/vdelecro/sage_patchbot/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 920, in compile_and_execute
exec(compiled, globs)
File "<doctest sage.geometry.lattice_polytope.LatticePolytopeClass.boundary_point_indices[4]>", line 1, in <module>
face.points()
File "/home/vdelecro/sage_patchbot/local/lib/python2.7/site-packages/sage/geometry/lattice_polytope.py", line 3680, in points
l = gcd(v)
NameError: global name 'gcd' is not defined
Yes i have see this one.
The last remaining problem is a SEGFAULT when running ppl_lattice_polytope.py doctests.
Could you post the log here?
Replying to @videlec:
Could you post the log here?
segfault_ppl_lattice_polytope.log
Short story is
sage: LatticePolytope_PPL((0,0),(1/2,1))
cause a Segfault.
Isn't it aleaxit/gmpy#174?
At least with the patch applied I got
sage: from sage.geometry.polyhedron.ppl_lattice_polytope import LatticePolytope_PPL
sage: LatticePolytope_PPL((0,0),(1/2,1))
Traceback (most recent call last):
...
TypeError: rational is not an integer
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
7596cff | remove sage/libs/ppl from doctree |
0cd326c | update checksum, snapshot version update in setup.py |
91b5d52 | Add path of gmpy2 in include_dirs |
0b965a7 | Trac #23024 Fix checksum (pplpy), spkg-install and spkg-check |
dd886fb | Trac #23024 Add new mpz gmpy2 to sage Integer conversion |
b7a5a21 | Trac #23024 SPKG.txt add upstream contact and licence. |
96146ca | Trac #23024 Deprecate sage.libs.ppl |
e85ae5f | Trac #23024 Use gmpy2 as a standard package |
da9aa35 | Trac #23024 Restore import gcd |
3cec632 | Trac #23024 : merge #22928 ppl.pyx add deprecation warning ... |
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
d206c14 | Trac #23024: ppl.pyx add deprecation warning to long doctest |
You are adding sage.libs.gmpy2 in 7596cff and removing it in 91b5d52...
Yes adding it was one of our old solutions. Before having a better include in pplpy's setup.py if i remember correctly. All of this historical things can be squashed later.
It's the same for ppl.pyx we don't need to show that we deleted it and restore and deprecate later.
There are a lot of trailing whitespaces that should be removed.
History rewrited. Trailing whitespaces removed in real_mpfr.pyx, complex_mpc.pyx and complex_number.pyx files.
I don't understand why you are modifying pplpy/SPKG.txt, pplpy/checksums.ini, pplpy/spkg-check, etc in both 886424b and a6e1cc7.
Build fine and all tests pass on my computer!
Description changed:
---
+++
@@ -1,4 +1,4 @@
-The Sage wrappers around [ppl](http://bugseng.com/products/ppl/) are now superseded by the standalone [pplpy](https://github.com/videlec/pplpy/) (Python/Cython compatible). This ticket proposes to replace `sage/src/libs/ppl.pyx` extension with pplpy. It
+The Sage wrappers around [ppl](http://bugseng.com/products/ppl/) are now superseded by the standalone [pplpy](https://github.com/videlec/pplpy/) (Python/Cython compatible). This ticket proposes to replace `src/sage/libs/ppl.pyx` extension with pplpy. It
- creates a new standard package pplpy
- upgrade gmpy2 from optional to standardUse sdh_pip_install instead of $PIP_INSTALL
Is the license of pplpy really GPL version 3? Or do you mean GPL version 3 or later? The difference is important.
This looks like a strange way to check if something is one:
Integer(p.divisor()).is_one()
What's wrong with
p.divisor() == 1
Replying to @jdemeyer:
Is the license of pplpy really
GPL version 3? Or do you meanGPL version 3 or later? The difference is important.
Yes the current one is GPL version 3 (see LICENSE.txt). Is it better to say GPL version 3 or later anyway ?
In backend_ppl.py you do from gmpy2 import mpz but you're not using it.
Replying to @jdemeyer:
What's wrong with
p.divisor() == 1
p.divisor() return a gmpy2.mpz.
Branch pushed to git repo; I updated commit sha1. New commits:
ef97c32 | Trac #23024: spkg-install `sdh_pip_install` instead ... |
Replying to @videlec:
...
What is wrong with>>> gmpy2.mpz(1) == 1 True
In this case you compare a python int with a gmpy2.mpz.
But with sage's Integer :
sage: gmpy2.mpz(1) == 1
False
Edit: Hum ok my bad.
Branch pushed to git repo; I updated commit sha1. New commits:
2f98d70 | Trac #23024: Code cleaning |
self-testing fails:
$ SAGE_CHECK=yes ./sage -f pplpy
...
[pplpy-0.7] Successfully installed pplpy-0.7
[pplpy-0.7] Cleaning up...
[pplpy-0.7]
[pplpy-0.7] real 0m38.345s
[pplpy-0.7] user 0m36.490s
[pplpy-0.7] sys 0m1.835s
[pplpy-0.7] Copying package files from temporary location /home/dima/Sage/sage-dev/local/var/tmp/sage/build/pplpy-0.7/inst to /home/dima/Sage/sage-dev/local
[pplpy-0.7] Running the test suite for pplpy-0.7...
[pplpy-0.7] running test
[pplpy-0.7] error: cannot copy tree './tests': not a directory
[pplpy-0.7]
This would still be a good thing to do IMO, in general.
Description changed:
---
+++
@@ -5,4 +5,4 @@
- replace the Sage code calling sage/libs/ppl.pyx to call pplpy
- make appropriate deprecation for `sage.libs.ppl`
-pplpy' Tarball : [pplpy-0.7.tar.gz](https://pypi.python.org/packages/ca/d3/1629e95997068c129e3a5235fd32a39e4fc39faca892cf9a2da11f842fb9/pplpy-0.7.tar.gz#md5=89dd6a1e75e06e287adae2ea4dbc143b)
+pplpy' Tarball : [pplpy-0.8.tar.gz](https://files.pythonhosted.org/packages/9b/bb/4973d00ec0b02bb5c4a744584a41d29b68bb9fec529407d81808e46adcca/pplpy-0.8.tar.gz)Description changed:
---
+++
@@ -1,4 +1,4 @@
-The Sage wrappers around [ppl](http://bugseng.com/products/ppl/) are now superseded by the standalone [pplpy](https://github.com/videlec/pplpy/) (Python/Cython compatible). This ticket proposes to replace `src/sage/libs/ppl.pyx` extension with pplpy. It
+The Sage wrappers around [ppl](http://bugseng.com/products/ppl/) are now superseded by the standalone [pplpy](https://gitlab.com/videlec/pplpy) (Python/Cython compatible). This ticket proposes to replace `src/sage/libs/ppl.pyx` extension with pplpy. It
- creates a new standard package pplpy
- upgrade gmpy2 from optional to standardBranch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
f5b7110 | Trac 23024: Add pplpy as a standard package |
82a5d24 | Trac 23024: Replace sage.libs.ppl with pplpy package |
59617e1 | Trac #23024: Use gmpy2 as a standard package |
8f51887 | Trac #23024: spkg-install `sdh_pip_install` instead ... |
d7a67d4 | Trac #23024: Code cleaning |
9ea9382 | Trac #23024: Remove HAVE_GMPY2 from coerce.pyx |
55ad161 | Trac #23024: fix ppl_lattice_polytope doctests |
Rebased on sage8.6
Description changed:
---
+++
@@ -5,4 +5,4 @@
- replace the Sage code calling sage/libs/ppl.pyx to call pplpy
- make appropriate deprecation for `sage.libs.ppl`
-pplpy' Tarball : [pplpy-0.8.tar.gz](https://files.pythonhosted.org/packages/9b/bb/4973d00ec0b02bb5c4a744584a41d29b68bb9fec529407d81808e46adcca/pplpy-0.8.tar.gz)
+pplpy' Tarball : [pplpy-0.8.1.tar.gz](https://files.pythonhosted.org/packages/b2/90/890f89536a0fdd66d78a07d963152502b4b1af7c94366d8bb16e7ff8e309/pplpy-0.8.1.tar.gz)upgrade pplpy to 0.8.1
Apparently a merge failure with sage 8.7.beta0
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
c37dcb0 | Trac 23024: Add pplpy as a standard package |
3f5eed8 | Trac 23024: Replace sage.libs.ppl with pplpy package |
26d3d54 | Trac #23024: Use gmpy2 as a standard package |
1a5d7e5 | Trac #23024: spkg-install `sdh_pip_install` instead ... |
62452d5 | Trac #23024: Code cleaning |
58a0e8a | Trac #23024: Remove HAVE_GMPY2 from coerce.pyx |
dabd865 | Trac #23024: fix ppl_lattice_polytope doctests |
9a06541 | Trac #23024: upgrade pplpy version to 0.8.1 |