upgrade rpy2 package 2.8.2 -> 3.3.5, upgrade R to 3.6.3, add new dependencies

Question

upgrade rpy2 package 2.8.2 -> 3.3.5, upgrade R to 3.6.3, add new dependencies

videlec opened this issue 6 years ago · 139 comments

Main changes:

https://pypi.org/project/rpy2/, supports python >= 3.6
cygwin.patch does not apply anymore
cffi>=1.13.1 is a new dependency, we install latest 1.14.0 (supports python >= 3.2)
pycparser is a new dependency (required by cffi, supports python >= 3.4)
https://pypi.org/project/tzlocal/ is a new dependency, we install latest 2.1 (supports python >= 2.7)
https://pypi.org/project/pytz/ is a new dependency, we update to latest 2020.1 (supports python >= 2.4)
add small patches to setup.py that
- disables printing on stdout (that perturbs pip) - accepted upstream
- removes the build dependency on pytest
some more dependencies (pytest, numpy, tzlocal) conditional on SAGE_CHECK!=no

tarballs: see checksums.ini (to download automatically, use ./configure --enable-download-from-upstream-url)

Upstream issues and PR

Depends on #29851
Depends on #30064
Depends on #30118
Depends on #30067
Depends on #30147
Depends on #29929
Depends on #30149

CC: @mkoeppe @EmmanuelCharpentier @timokau @isuruf @slel @antonio-rojas @embray @dimpase

Component: packages: standard

Author: Vincent Delecroix, Matthias Koeppe

Branch: 54d0f99

Reviewer: Emmanuel Charpentier, Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29441

Answer 1 · 2020-03-31T20:50:47.000Z

comment:1

The cygwin.patch does not longer apply.

Answer 2 · 2020-03-31T20:50:47.000Z

Description changed:

--- 
+++ 
@@ -1,3 +1,3 @@
-The release 3.2.7 is only Python 3 compatible
+The release 3.2.7 is only Python 3 compatible.
 
 tarball: https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz

Answer 3 · 2020-03-31T20:59:34.000Z

Description changed:

--- 
+++ 
@@ -1,3 +1,11 @@
 The release 3.2.7 is only Python 3 compatible.
 
-tarball: https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
+Main changes:
+- `cygwin.patch` does not apply anymore
+- [https://pypi.org/project/pytest/](https://pypi.org/project/pytest/) is a new dependency (!?)
+- [https://pypi.org/project/cffi/ cffi>=1.13.1](https://pypi.org/project/cffi/ cffi>=1.13.1) is a new dependency
+
+tarballs: 
+- https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
+- https://files.pythonhosted.org/packages/6d/4e/572aed20422dee7fa2bd27995b2a53a32de90c1826e5531c9df6d3ea77ed/pytest-5.4.1.tar.gz
+- https://files.pythonhosted.org/packages/05/54/3324b0c46340c31b909fcec598696aaec7ddc8c18a63f2db352562d3354c/cffi-1.14.0.tar.gz

Answer 4 · 2020-03-31T21:01:45.000Z

Description changed:

--- 
+++ 
@@ -4,8 +4,10 @@
 - `cygwin.patch` does not apply anymore
 - [https://pypi.org/project/pytest/](https://pypi.org/project/pytest/) is a new dependency (!?)
 - [https://pypi.org/project/cffi/ cffi>=1.13.1](https://pypi.org/project/cffi/ cffi>=1.13.1) is a new dependency
+- [https://pypi.org/project/pycparser/ pycparser](https://pypi.org/project/pycparser/ pycparser) is a new dependency (required by cffi)
 
 tarballs: 
 - https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
 - https://files.pythonhosted.org/packages/6d/4e/572aed20422dee7fa2bd27995b2a53a32de90c1826e5531c9df6d3ea77ed/pytest-5.4.1.tar.gz
 - https://files.pythonhosted.org/packages/05/54/3324b0c46340c31b909fcec598696aaec7ddc8c18a63f2db352562d3354c/cffi-1.14.0.tar.gz
+- https://files.pythonhosted.org/packages/0f/86/e19659527668d70be91d0369aeaa055b4eb396b0f387a4f92293a20035bd/pycparser-2.20.tar.gz

Answer 5 · 2020-03-31T21:05:12.000Z

comment:4

pytest has a lot of dependencies

attrs
importlib-metadata
more-itertools
pluggy
py
zipp
The easiest might be to patch rpy2 to not depend on pytest. I suspect that only the test suite of rpy2 depends on pytest.

Answer 6 · 2020-03-31T21:13:26.000Z

Description changed:

--- 
+++ 
@@ -2,12 +2,15 @@
 
 Main changes:
 - `cygwin.patch` does not apply anymore
+- the `setup.py` script is broken in several ways (!!)
 - [https://pypi.org/project/pytest/](https://pypi.org/project/pytest/) is a new dependency (!?)
 - [https://pypi.org/project/cffi/ cffi>=1.13.1](https://pypi.org/project/cffi/ cffi>=1.13.1) is a new dependency
 - [https://pypi.org/project/pycparser/ pycparser](https://pypi.org/project/pycparser/ pycparser) is a new dependency (required by cffi)
+- [https://pypi.org/project/tzlocal/ tzlocal](https://pypi.org/project/tzlocal/ tzlocal)
 
 tarballs: 
 - https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
 - https://files.pythonhosted.org/packages/6d/4e/572aed20422dee7fa2bd27995b2a53a32de90c1826e5531c9df6d3ea77ed/pytest-5.4.1.tar.gz
 - https://files.pythonhosted.org/packages/05/54/3324b0c46340c31b909fcec598696aaec7ddc8c18a63f2db352562d3354c/cffi-1.14.0.tar.gz
 - https://files.pythonhosted.org/packages/0f/86/e19659527668d70be91d0369aeaa055b4eb396b0f387a4f92293a20035bd/pycparser-2.20.tar.gz
+- https://files.pythonhosted.org/packages/c6/52/5ec375d4efcbe4e31805f3c4b301bdfcff9dcbdb3605d4b79e117a16b38d/tzlocal-2.0.0.tar.gz

Answer 7 · 2020-03-31T21:14:39.000Z

comment:6

Good news: tests pass on my machine!

Answer 8 · 2020-03-31T21:18:16.000Z

comment:8

See #28998 for pytest

Answer 9 · 2020-03-31T21:20:16.000Z

comment:9

Replying to @videlec:

pytest has a lot of dependencies

attrs

importlib-metadata

more-itertools

pluggy

py

zipp

Tall order, indeed...

The easiest might be to patch rpy2 to not depend on pytest. I suspect that only the test suite of rpy2 depends on pytest.

My first reaction was: No, no, no, no, no. And no.". A patched version has a very heavy price in terms of maintenance each and every update entails patch checking/upgrading, up to the point the whole mess becomes unmaintainable.

The maintenance price of the added dependencies should be considered, of course, but I suspect that it may be lower than the price of a patch to rpy2.

Answer 10 · 2020-03-31T21:20:16.000Z

Description changed:

--- 
+++ 
@@ -2,15 +2,12 @@
 
 Main changes:
 - `cygwin.patch` does not apply anymore
-- the `setup.py` script is broken in several ways (!!)
 - [https://pypi.org/project/pytest/](https://pypi.org/project/pytest/) is a new dependency (!?)
 - [https://pypi.org/project/cffi/ cffi>=1.13.1](https://pypi.org/project/cffi/ cffi>=1.13.1) is a new dependency
 - [https://pypi.org/project/pycparser/ pycparser](https://pypi.org/project/pycparser/ pycparser) is a new dependency (required by cffi)
-- [https://pypi.org/project/tzlocal/ tzlocal](https://pypi.org/project/tzlocal/ tzlocal)
 
 tarballs: 
 - https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
 - https://files.pythonhosted.org/packages/6d/4e/572aed20422dee7fa2bd27995b2a53a32de90c1826e5531c9df6d3ea77ed/pytest-5.4.1.tar.gz
 - https://files.pythonhosted.org/packages/05/54/3324b0c46340c31b909fcec598696aaec7ddc8c18a63f2db352562d3354c/cffi-1.14.0.tar.gz
 - https://files.pythonhosted.org/packages/0f/86/e19659527668d70be91d0369aeaa055b4eb396b0f387a4f92293a20035bd/pycparser-2.20.tar.gz
-- https://files.pythonhosted.org/packages/c6/52/5ec375d4efcbe4e31805f3c4b301bdfcff9dcbdb3605d4b79e117a16b38d/tzlocal-2.0.0.tar.gz

Answer 11 · 2020-03-31T21:22:24.000Z

comment:10

Replying to @mkoeppe:

See #28998 for pytest

Thanks for the pointer. Would pytest be installed as a "standard" package?

Answer 12 · 2020-03-31T21:23:03.000Z

comment:11

Replying to @mkoeppe:

See #28998 for pytest

Dors it need much work ?

Answer 13 · 2020-03-31T21:25:56.000Z

comment:12

Replying to @videlec:

Replying to @mkoeppe:

See #28998 for pytest

Thanks for the pointer. Would pytest be installed as a "standard" package?

It would have to : r depends on rpy2, which would depend on pytest. ISTR that you yourself stated that having a standard package depend on an optional package would be madness (and I agree...). Therefore...

Answer 14 · 2020-03-31T21:26:51.000Z

New commits:

`de753f8`	`upgrade rpy2 version`
`7ca1336`	`remove cygwin.patch (rpy2)`
`c525fc2`	`add setup.patch (rpy2)`

Answer 15 · 2020-03-31T21:26:51.000Z

Branch: public/29441

Answer 16 · 2020-03-31T21:26:51.000Z

Commit: c525fc2

Answer 17 · 2020-03-31T21:28:28.000Z

comment:14

As far as tests are concerned, the design of rpy2 is weird. Tests should be separate from the rpy2 module (and hence the dependency of pytest superfluous).

Answer 18 · 2020-03-31T21:37:30.000Z

comment:15

Replying to @EmmanuelCharpentier:

Milestone changed from sage-9.2 to sage-9.1

Was this intentional?
We definitely cannot drop py2 support for 9.1

Answer 19 · 2020-03-31T21:38:44.000Z

comment:16

Replying to @mkoeppe:

Replying to @EmmanuelCharpentier:

Milestone changed from sage-9.2 to sage-9.1

Was this intentional?
We definitely cannot drop py2 support for 9.1

Agreed. This was a mistake.

Answer 20 · 2020-03-31T21:40:20.000Z

Description changed:

--- 
+++ 
@@ -2,12 +2,15 @@
 
 Main changes:
 - `cygwin.patch` does not apply anymore
-- [https://pypi.org/project/pytest/](https://pypi.org/project/pytest/) is a new dependency (!?)
-- [https://pypi.org/project/cffi/ cffi>=1.13.1](https://pypi.org/project/cffi/ cffi>=1.13.1) is a new dependency
-- [https://pypi.org/project/pycparser/ pycparser](https://pypi.org/project/pycparser/ pycparser) is a new dependency (required by cffi)
+- [pytest](https://pypi.org/project/pytest/) is a new dependency (!?)
+- [cffi>=1.13.1](https://pypi.org/project/cffi/) is a new dependency
+- [pycparser](https://pypi.org/project/pycparser/) is a new dependency (required by cffi)
 
 tarballs: 
 - https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
 - https://files.pythonhosted.org/packages/6d/4e/572aed20422dee7fa2bd27995b2a53a32de90c1826e5531c9df6d3ea77ed/pytest-5.4.1.tar.gz
 - https://files.pythonhosted.org/packages/05/54/3324b0c46340c31b909fcec598696aaec7ddc8c18a63f2db352562d3354c/cffi-1.14.0.tar.gz
 - https://files.pythonhosted.org/packages/0f/86/e19659527668d70be91d0369aeaa055b4eb396b0f387a4f92293a20035bd/pycparser-2.20.tar.gz
+
+upstream issues and PR
+- [issues #670 (about tests in rpy2 and pytest dependency)](https://github.com/rpy2/rpy2/issues/670)

Answer 21 · 2020-03-31T21:40:54.000Z

comment:18

Replying to @videlec:

As far as tests are concerned, the design of rpy2 is weird. Tests should be separate from the rpy2 module (and hence the dependency of pytest superfluous).

Replying to @videlec:

As far as tests are concerned, the design of rpy2 is weird. Tests should be separate from the rpy2 module (and hence the dependency of pytest superfluous).

Can rpy2 use packages hypothetically installed by an hypothetical package implementing #28998 ?
should we instead wait for an upstream-fixed rpy2 (supposing that upstream agrees with you...) ?

Answer 22 · 2020-03-31T21:42:58.000Z

comment:19

Replying to @EmmanuelCharpentier:

Replying to @videlec:

As far as tests are concerned, the design of rpy2 is weird. Tests should be separate from the rpy2 module (and hence the dependency of pytest superfluous).

should we instead wait for an upstream-fixed rpy2 (supposing that upstream agrees with you...) ?

YES

Answer 23 · 2020-03-31T21:43:31.000Z

Description changed:

--- 
+++ 
@@ -2,15 +2,12 @@
 
 Main changes:
 - `cygwin.patch` does not apply anymore
-- [pytest](https://pypi.org/project/pytest/) is a new dependency (!?)
-- [cffi>=1.13.1](https://pypi.org/project/cffi/) is a new dependency
-- [pycparser](https://pypi.org/project/pycparser/) is a new dependency (required by cffi)
+- [https://pypi.org/project/pytest/](https://pypi.org/project/pytest/) is a new dependency (!?)
+- [https://pypi.org/project/cffi/ cffi>=1.13.1](https://pypi.org/project/cffi/ cffi>=1.13.1) is a new dependency
+- [https://pypi.org/project/pycparser/ pycparser](https://pypi.org/project/pycparser/ pycparser) is a new dependency (required by cffi)
 
 tarballs: 
 - https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
 - https://files.pythonhosted.org/packages/6d/4e/572aed20422dee7fa2bd27995b2a53a32de90c1826e5531c9df6d3ea77ed/pytest-5.4.1.tar.gz
 - https://files.pythonhosted.org/packages/05/54/3324b0c46340c31b909fcec598696aaec7ddc8c18a63f2db352562d3354c/cffi-1.14.0.tar.gz
 - https://files.pythonhosted.org/packages/0f/86/e19659527668d70be91d0369aeaa055b4eb396b0f387a4f92293a20035bd/pycparser-2.20.tar.gz
-
-upstream issues and PR
-- [issues #670 (about tests in rpy2 and pytest dependency)](https://github.com/rpy2/rpy2/issues/670)

Answer 24 · 2020-03-31T21:43:31.000Z

comment:20

rpy2 should be patched so that pytest is not an actual dependency, but test-only.

rpy2 is missing modern metadata such as requirements.txt, test-requirements.txt, tox.ini where things like this are commonly declared.

#28998 would install pytest only when SAGE_CHECK=yes.

Answer 25 · 2020-03-31T21:45:01.000Z

comment:21

By the way, I recommend that you add the new upstream_url fields to checksum.ini.
Makes it easier to test the ticket.

Answer 26 · 2020-03-31T21:45:49.000Z

comment:22

(See #29425 for an example)

Answer 27 · 2020-03-31T21:47:20.000Z

comment:23

Replying to @mkoeppe:

Description modified (diff)

@mkoeppe: Did you intentionally erase my modifications to the ticket description!?

Answer 28 · 2020-03-31T21:49:34.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`e672b6e`	`add upstream_url field to rpy2 checksums.ini`

Answer 29 · 2020-03-31T21:49:34.000Z

Changed commit from c525fc2 to e672b6e

Answer 30 · 2020-03-31T21:52:22.000Z

Description changed:

--- 
+++ 
@@ -2,12 +2,13 @@
 
 Main changes:
 - `cygwin.patch` does not apply anymore
-- [https://pypi.org/project/pytest/](https://pypi.org/project/pytest/) is a new dependency (!?)
-- [https://pypi.org/project/cffi/ cffi>=1.13.1](https://pypi.org/project/cffi/ cffi>=1.13.1) is a new dependency
-- [https://pypi.org/project/pycparser/ pycparser](https://pypi.org/project/pycparser/ pycparser) is a new dependency (required by cffi)
+- [cffi>=1.13.1](https://pypi.org/project/cffi/) is a new dependency
+- [pycparser](https://pypi.org/project/pycparser/) is a new dependency (required by cffi)
 
 tarballs: 
 - https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
-- https://files.pythonhosted.org/packages/6d/4e/572aed20422dee7fa2bd27995b2a53a32de90c1826e5531c9df6d3ea77ed/pytest-5.4.1.tar.gz
 - https://files.pythonhosted.org/packages/05/54/3324b0c46340c31b909fcec598696aaec7ddc8c18a63f2db352562d3354c/cffi-1.14.0.tar.gz
 - https://files.pythonhosted.org/packages/0f/86/e19659527668d70be91d0369aeaa055b4eb396b0f387a4f92293a20035bd/pycparser-2.20.tar.gz
+
+Upstream issues and PR
+- https://github.com/rpy2/rpy2/issues/670

Answer 31 · 2020-03-31T21:59:04.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`86d0d17`	`let rpy2 not depend on pytest`

Answer 32 · 2020-03-31T21:59:04.000Z

Changed commit from e672b6e to 86d0d17

Answer 33 · 2020-03-31T22:01:02.000Z

comment:27

Replying to @videlec:

Replying to @mkoeppe:

Description modified (diff)

@mkoeppe: Did you intentionally erase my modifications to the ticket description!?

No, sorry! Looks like on this ticket trac is playing funny tricks

Answer 34 · 2020-03-31T22:21:09.000Z

Changed commit from 86d0d17 to 5eee83e

Answer 35 · 2020-03-31T22:21:09.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`a1debab`	`cffi package (rpy2 dependency)`
`a8b6e66`	`pycparser (rpy2 dependency)`
`5eee83e`	`update rpy2 dependencies`

Answer 36 · 2020-03-31T22:23:15.000Z

comment:29

Ready for tests (especially on cygwin), but not for merging!

Answer 37 · 2020-03-31T22:28:12.000Z

comment:30

If you want to test cygwin via github actions, be sure to use #29403

Answer 38 · 2020-04-19T21:24:34.000Z

comment:32

upstream is not (totally) convinced by dropping the pytest dependency. I propose to however keep the patch in Sage (on archlinux rpy2 does not depend on pytest either).

I will add a commit that still installs the test as part of rpy2 (it did not make much sense to remove it).

Answer 39 · 2020-04-20T07:37:35.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`308d39c`	`still install tests along rpy2`

Answer 40 · 2020-04-20T07:37:35.000Z

Changed commit from 5eee83e to 308d39c

Answer 41 · 2020-04-20T08:37:23.000Z

Author: Vincent Delecroix

Answer 42 · 2020-04-20T08:37:23.000Z

Description changed:

--- 
+++ 
@@ -4,6 +4,9 @@
 - `cygwin.patch` does not apply anymore
 - [cffi>=1.13.1](https://pypi.org/project/cffi/) is a new dependency
 - [pycparser](https://pypi.org/project/pycparser/) is a new dependency (required by cffi)
+- add a small patch to `setup.py` that
+  - disables printing on `stdout` (that perturbs `pip`)
+  - removes the dependency to `pytest`
 
 tarballs: 
 - https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz

Answer 43 · 2020-04-23T03:19:23.000Z

comment:35

Needs rebasing

Answer 44 · 2020-06-14T20:03:35.000Z

comment:36

#29813 adds pytest

Answer 45 · 2020-06-14T20:04:28.000Z

Description changed:

--- 
+++ 
@@ -8,6 +8,8 @@
   - disables printing on `stdout` (that perturbs `pip`)
   - removes the dependency to `pytest`
 
+https://pypi.org/project/rpy2/
+
 tarballs: 
 - https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
 - https://files.pythonhosted.org/packages/05/54/3324b0c46340c31b909fcec598696aaec7ddc8c18a63f2db352562d3354c/cffi-1.14.0.tar.gz

Answer 46 · 2020-06-14T20:04:39.000Z

comment:38

Latest rpy2 is now 3.3.3

Answer 47 · 2020-07-03T18:45:33.000Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`68ea7be`	`upgrade rpy2 version`
`f1cd4e2`	`remove cygwin.patch (rpy2)`
`68f6f92`	`add setup.patch (rpy2)`
`b550f43`	`add upstream_url field to rpy2 checksums.ini`
`2cd2191`	`let rpy2 not depend on pytest`
`b76ee25`	`cffi package (rpy2 dependency)`
`84ded30`	`pycparser (rpy2 dependency)`
`2e0fe73`	`update rpy2 dependencies`
`81f580e`	`still install tests along rpy2`

Answer 48 · 2020-07-03T18:45:33.000Z

Changed commit from 308d39c to 81f580e

Answer 49 · 2020-07-03T18:45:53.000Z

comment:40

Rebased on 9.2.beta2

Answer 50 · 2020-07-03T18:50:28.000Z

Description changed:

--- 
+++ 
@@ -1,19 +1,15 @@
-The release 3.2.7 is only Python 3 compatible.
-
 Main changes:
+- [https://pypi.org/project/rpy2/](https://pypi.org/project/rpy2/), supports python >= 3.6
 - `cygwin.patch` does not apply anymore
-- [cffi>=1.13.1](https://pypi.org/project/cffi/) is a new dependency
-- [pycparser](https://pypi.org/project/pycparser/) is a new dependency (required by cffi)
+- [cffi>=1.13.1](https://pypi.org/project/cffi/) is a new dependency, we install latest 1.14.0 (supports python >= 3.2)
+- [pycparser](https://pypi.org/project/pycparser/) is a new dependency (required by cffi, supports python >= 3.4)
 - add a small patch to `setup.py` that
   - disables printing on `stdout` (that perturbs `pip`)
   - removes the dependency to `pytest`
 
-https://pypi.org/project/rpy2/
 
-tarballs: 
-- https://files.pythonhosted.org/packages/39/c0/61120f9dae06b4887426d229b68a7a5f0ca1f9cb3986319bb9484819a28d/rpy2-3.2.7.tar.gz
-- https://files.pythonhosted.org/packages/05/54/3324b0c46340c31b909fcec598696aaec7ddc8c18a63f2db352562d3354c/cffi-1.14.0.tar.gz
-- https://files.pythonhosted.org/packages/0f/86/e19659527668d70be91d0369aeaa055b4eb396b0f387a4f92293a20035bd/pycparser-2.20.tar.gz
+tarballs: see checksums.ini
 
 Upstream issues and PR
 - https://github.com/rpy2/rpy2/issues/670
+

Answer 51 · 2020-07-03T19:03:24.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`d4c1672`	`build/pkgs: Upgrade rpy2 to 3.3.4, use patterns for upstream_urls of dependencies`

Answer 52 · 2020-07-03T19:03:24.000Z

Changed commit from 81f580e to d4c1672

Answer 53 · 2020-07-03T19:08:51.000Z

comment:44

Patches need updating!

Answer 54 · 2020-07-03T19:08:51.000Z

Description changed:

--- 
+++ 
@@ -8,7 +8,7 @@
   - removes the dependency to `pytest`
 
 
-tarballs: see checksums.ini
+tarballs: see `checksums.ini` (to download automatically, use `./configure --enable-download-from-upstream-url`)
 
 Upstream issues and PR
 - https://github.com/rpy2/rpy2/issues/670

Answer 55 · 2020-07-03T19:55:13.000Z

comment:45

rpy2/rpy2#716

Answer 56 · 2020-07-04T04:33:05.000Z

Changed author from Vincent Delecroix to Vincent Delecroix, Matthias Koeppe

Answer 57 · 2020-07-04T04:33:05.000Z

Dependencies: #29813

Answer 58 · 2020-07-04T04:41:45.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`28552a2`	`Update patches`
`04b4c67`	`build/pkgs/pytest: New`
`55babfa`	`src/bin/sage [-i]: Set SAGE_CHECK here so that Makefile dependencies can depend on it`
`bedc7ae`	`build/make/Makefile.in: Allow pip packages as dependencies`
`e0db954`	`Merge branch 't/29813/add_pytest_as_a_type_optional__source_pip_package' into t/29441/public/29441`
`20133ad`	`build/pkgs/rpy2: Add spkg-check.in, add conditional dep on pytest, simplify spkg-install.in`
`c02362e`	`build/pkgs/tzlocal: New (pip package)`
`d33797f`	`build/pkgs/rpy2/dependencies: Add conditional dep on tzlocal, numpy`

Answer 59 · 2020-07-04T04:41:45.000Z

Changed commit from d4c1672 to d33797f

Answer 60 · 2020-07-04T04:43:05.000Z

comment:48

Replying to @EmmanuelCharpentier:

Replying to @videlec:

As far as tests are concerned, the design of rpy2 is weird. Tests should be separate from the rpy2 module (and hence the dependency of pytest superfluous).

Can rpy2 use packages hypothetically installed by an hypothetical package implementing #28998 ?

Yes, this is done here on the ticket now, using #29813 instead of the larger #28998.

Answer 61 · 2020-07-04T04:47:33.000Z

comment:50

Build testing runs at https://github.com/mkoeppe/sage/actions/runs/157245915

Answer 62 · 2020-07-04T04:48:48.000Z

Description changed:

--- 
+++ 
@@ -3,8 +3,8 @@
 - `cygwin.patch` does not apply anymore
 - [cffi>=1.13.1](https://pypi.org/project/cffi/) is a new dependency, we install latest 1.14.0 (supports python >= 3.2)
 - [pycparser](https://pypi.org/project/pycparser/) is a new dependency (required by cffi, supports python >= 3.4)
-- add a small patch to `setup.py` that
-  - disables printing on `stdout` (that perturbs `pip`)
+- add small patches to `setup.py` that
+  - disables printing on `stdout` (that perturbs `pip`) - accepted upstream
   - removes the dependency to `pytest`
 
 
@@ -12,4 +12,4 @@
 
 Upstream issues and PR
 - https://github.com/rpy2/rpy2/issues/670
-
+- https://github.com/rpy2/rpy2/pull/716

Answer 63 · 2020-07-04T04:50:00.000Z

Description changed:

--- 
+++ 
@@ -5,8 +5,8 @@
 - [pycparser](https://pypi.org/project/pycparser/) is a new dependency (required by cffi, supports python >= 3.4)
 - add small patches to `setup.py` that
   - disables printing on `stdout` (that perturbs `pip`) - accepted upstream
-  - removes the dependency to `pytest`
-
+  - removes the build dependency on `pytest`
+- some more dependencies (pytest, numpy, tzlocal) conditional on SAGE_CHECK!=no
 
 tarballs: see `checksums.ini` (to download automatically, use `./configure --enable-download-from-upstream-url`)

Answer 64 · 2020-07-04T15:46:32.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`b66e9df`	`build/pkgs/{cffi,pycparser}/dependencies: New`

Answer 65 · 2020-07-04T15:46:32.000Z

Changed commit from d33797f to b66e9df

Answer 66 · 2020-07-08T15:33:09.000Z

Changed dependencies from #29813 to none

Answer 67 · 2020-07-08T15:35:29.000Z

Changed commit from b66e9df to 4c7f18d

Answer 68 · 2020-07-08T15:35:29.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`539c182`	`build/make/install: Do not depend on src/bin/sage-version.sh`
`761092c`	`Merge branch 't/29987/build_make_install__do_not_depend_on_src_bin_sage_version_sh' into t/30064/fix_tox_docker_builds_broken_by__29884`
`f2efa6a`	`src/doc/bootstrap: Create the directory src/doc/en/reference/repl if it does not exist`
`b7bf43b`	`build/bin/write-dockerfile.sh: ADD src/bin for bootstrapping, needed by src/doc/bootstrap after #29884`
`365ce61`	`Merge branch 'u/mkoeppe/fix_tox_docker_builds_broken_by__29884' of git://trac.sagemath.org/sage into HEAD`
`1e7becc`	`tox.ini [debian-buster, -sid]: IGNORE_MISSING_SYSTEM_PACKAGES=yes because of libpython3.7-dev`
`fb61a31`	`Merge branch 'u/mkoeppe/tox_ini__debian_bullseye___sid_have_python3_8_instead_of_3_7' of git://trac.sagemath.org/sage into 9.2.beta3+ci-fixes`
`4c7f18d`	`Merge branch '9.2.beta3+ci-fixes' into t/29441/public/29441`

Answer 69 · 2020-07-08T15:37:50.000Z

Dependencies: #29851, #30064

Answer 70 · 2020-07-08T15:42:00.000Z

comment:57

Replying to @mkoeppe:

Build testing runs at https://github.com/mkoeppe/sage/actions/runs/157245915

Some unrelated problems are in the way. For example, on ubuntu-eoan-minimal,

  [pytest]     Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/pytest/
  [pytest]     Could not fetch URL https://pypi.org/simple/pytest/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max

Answer 71 · 2020-07-12T05:04:19.000Z

Changed dependencies from #29851, #30064 to #29851, #30064, #30118

Answer 72 · 2020-07-12T05:11:03.000Z

Changed commit from 4c7f18d to 3b14bf5

Answer 73 · 2020-07-12T05:11:03.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`25393b0`	`Handle SAGE_CHECK_PACKAGES in build/make/Makefile.in, not sage-spkg`
`833ff0e`	`Merge branch 't/30118/handle_sage_check_packages_in_build_make_makefile_in__not_sage_spkg' into t/29441/public/29441`
`3b14bf5`	`build/pkgs/rpy2/dependencies: Conditionalize on SAGE_CHECK_rpy2`

Answer 74 · 2020-07-16T13:44:48.000Z

comment:61

On Debian testing running n core i7 + 16 GB RAM
Sage configured to use systemwide R
Rebase on develop at 9.2.beta5

sage: r.sessionInfo()
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-24-69f706c2bf29> in <module>()
----> 1 r.sessionInfo()

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/interfaces/r.py in __call__(self, *args, **kwds)
   2001             [1] 3
   2002         """
-> 2003         return self._parent.function_call(self._name, args=list(args), kwds=kwds)
   2004 
   2005 def is_RElement(x):

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/interfaces/r.py in function_call(self, function, args, kwds)
   1078         self._check_valid_function_name(function)
   1079         return self.new("%s(%s)"%(function, ",".join([s.name() for s in args] +
-> 1080                                                      [self._sage_to_r_name(key)+'='+kwds[key].name() for key in kwds ] )))
   1081 
   1082     def call(self, function_name, *args, **kwds):

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/interfaces/interface.py in new(self, code)
    368 
    369     def new(self, code):
--> 370         return self(code)
    371 
    372     ###################################################################

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/interfaces/interface.py in __call__(self, x, name)
    294 
    295         if isinstance(x, str):
--> 296             return cls(self, x, name=name)
    297         try:
    298             # Special methods do not and should not have an option to

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/interfaces/interface.py in __init__(self, parent, value, is_name, name)
    716         else:
    717             try:
--> 718                 self._name = parent._create(value, name=name)
    719             except (TypeError, RuntimeError, ValueError) as x:
    720                 raise TypeError(x)

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/interfaces/interface.py in _create(self, value, name)
    499     def _create(self, value, name=None):
    500         name = self._next_var_name() if name is None else name
--> 501         self.set(name, value)
    502         return name
    503 

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/interfaces/r.py in set(self, var, value)
   1124         """
   1125         cmd = '%s <- %s'%(var,value)
-> 1126         out = self.eval(cmd)
   1127 
   1128     def get(self, var):

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/interfaces/r.py in eval(self, code, *args, **kwds)
   1337         """
   1338         self._lazy_init()
-> 1339         return str(robjects.r(code)).rstrip()
   1340 
   1341 

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/misc/lazy_import.pyx in sage.misc.lazy_import.LazyImport.__getattr__ (build/cythonized/sage/misc/lazy_import.c:3525)()
    319             True
    320         """
--> 321         return getattr(self.get_object(), attr)
    322 
    323     # We need to wrap all the slot methods, as they are not forwarded

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/misc/lazy_import.pyx in sage.misc.lazy_import.LazyImport.get_object (build/cythonized/sage/misc/lazy_import.c:2336)()
    186         if likely(self._object is not None):
    187             return self._object
--> 188         return self._get_object()
    189 
    190     cpdef _get_object(self):

/usr/local/sage-9/local/lib/python3.7/site-packages/sage/misc/lazy_import.pyx in sage.misc.lazy_import.LazyImport._get_object (build/cythonized/sage/misc/lazy_import.c:2575)()
    218         elif self._at_startup and not startup_guard:
    219             print('Option ``at_startup=True`` for lazy import {0} not needed anymore'.format(self._name))
--> 220         self._object = getattr(__import__(self._module, {}, {}, [self._name]), self._name)
    221         name = self._as_name
    222         if self._deprecation is not None:

/usr/local/sage-9/local/lib/python3.7/site-packages/rpy2/robjects/__init__.py in <module>()
     24 
     25 from . import conversion
---> 26 from . import vectors
     27 from . import language
     28 

/usr/local/sage-9/local/lib/python3.7/site-packages/rpy2/robjects/vectors.py in <module>()
     14 import time
     15 import pytz
---> 16 import tzlocal
     17 from datetime import date, datetime, timedelta, timezone
     18 from time import struct_time, mktime, tzname

ModuleNotFoundError: No module named 'tzlocal'

All othetr attemps end up similarly.

==>needs_work

Answer 75 · 2020-07-16T14:09:46.000Z

Attachment: rdoctest.txt.gz

Doctest for src/sage/interfaces/r.py

Answer 76 · 2020-07-16T14:11:09.000Z

comment:62

Replying to @EmmanuelCharpentier:

On Debian testing running n core i7 + 16 GB RAM

Sage configured to use systemwide R

Rebase on develop at 9.2.beta5

[ Snip... ]

BTW, sage -t --long src/sage/interfaces/r.py gives the enclosed rdoctest.txt.gz

HTH,

Answer 77 · 2020-07-16T15:32:40.000Z

comment:63

yeah, I see this error too.

Answer 78 · 2020-07-16T15:39:09.000Z

comment:64

missing dep, can be installed using pip:

$ ./sage --pip install tzlocal
Collecting tzlocal
  Downloading tzlocal-2.1-py2.py3-none-any.whl (16 kB)
Requirement already satisfied: pytz in ./local/lib/python3.7/site-packages (from tzlocal) (2018.7)
Installing collected packages: tzlocal
Successfully installed tzlocal-2.1
dimpase@penguin:~/sage$ ./sage
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 9.2.beta5, Release Date: 2020-07-12               │
│ Using Python 3.7.3. Type "help()" for help.                        │
└────────────────────────────────────────────────────────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Warning: this is a prerelease version, and it may be unstable.     ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
sage: r.sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 10 (buster)

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas/liblapack.so.3

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] tools     stats     graphics  grDevices utils     datasets  methods  
[8] base     

loaded via a namespace (and not attached):
[1] compiler_3.5.2

Answer 79 · 2020-07-16T16:42:54.000Z

comment:65

Replying to @dimpase:

missing dep, can be installed using pip:

[ Snip... ]

Could you add the relevant commit(s) to this branch before a new test ?

Answer 80 · 2020-07-16T17:00:03.000Z

comment:66

this seems to need addition of a new pip package, no?

Answer 81 · 2020-07-16T17:03:46.000Z

comment:67

oops, the package is there, it's just not getting installed for some reason, I guess
due to something fishy in

--- a/build/pkgs/rpy2/dependencies
+++ b/build/pkgs/rpy2/dependencies
@@ -1,4 +1,4 @@
-$(PYTHON) r six | $(PYTHON_TOOLCHAIN)
+$(PYTHON) r cffi | $(PYTHON_TOOLCHAIN) pycparser $(and $(filter-out no,$(SAGE_CHECK_rpy2)), pytest tzlocal numpy)

Answer 82 · 2020-07-16T17:11:28.000Z

comment:68

tzlocal is only pulled in for tests (SAGE_CHECK=yes). Is it needed for normal installation or at runtime as well?

Answer 83 · 2020-07-16T17:12:50.000Z

comment:69

In that case, we would have to add it as a normal package.

Answer 84 · 2020-07-16T17:14:21.000Z

comment:70

... and update package pytz to 2020.1

Answer 85 · 2020-07-16T17:16:27.000Z

comment:71

Looking at the traceback in comment 61, the answer seems to be "yes".

I'll prepare the pytz update first.

Answer 86 · 2020-07-16T17:21:50.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`5f93e7f`	`build/pkgs/pytz: Update to 2020.1`

Answer 87 · 2020-07-16T17:21:50.000Z

Changed commit from 3b14bf5 to 5f93e7f

Answer 88 · 2020-07-16T17:27:21.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`091ed90`	`build/pkgs/tzlocal: Make it a normal, standard package (new dep of standard package rpy2)`

Answer 89 · 2020-07-16T17:27:21.000Z

Changed commit from 5f93e7f to 091ed90

Answer 90 · 2020-07-16T17:28:18.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`b01e92d`	`build/pkgs/rpy2/dependencies: Add tzlocal pytz as normal dependencies`

Answer 91 · 2020-07-16T17:28:18.000Z

Changed commit from 091ed90 to b01e92d

Answer 92 · 2020-07-16T17:30:36.000Z

Description changed:

--- 
+++ 
@@ -3,6 +3,9 @@
 - `cygwin.patch` does not apply anymore
 - [cffi>=1.13.1](https://pypi.org/project/cffi/) is a new dependency, we install latest 1.14.0 (supports python >= 3.2)
 - [pycparser](https://pypi.org/project/pycparser/) is a new dependency (required by cffi, supports python >= 3.4)
+- [https://pypi.org/project/tzlocal/](https://pypi.org/project/tzlocal/) is a new dependency, we install latest 2.1 (supports python >= 2.7)
+- [https://pypi.org/project/pytz/](https://pypi.org/project/pytz/) is a new dependency, we update to latest 2020.1 (supports python >= 2.4)
+
 - add small patches to `setup.py` that
   - disables printing on `stdout` (that perturbs `pip`) - accepted upstream
   - removes the build dependency on `pytest`

Answer 93 · 2020-07-16T17:49:48.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`4ce8970`	`build/pkgs/rpy2: Update to 3.3.5`

Answer 94 · 2020-07-16T17:49:48.000Z

Changed commit from b01e92d to 4ce8970

Answer 95 · 2020-07-16T17:51:08.000Z

comment:78

3.3.5 has a bugfix but did not backport rpy2/rpy2#716, so we have to keep it as a patch.

Answer 96 · 2020-07-16T18:00:31.000Z

comment:79

Replying to @sagetrac-git:

Branch pushed to git repo; I updated commit sha1. New commits:

`4ce8970`	`build/pkgs/rpy2: Update to 3.3.5`

The mails emanating from Trac are not timely. So please don't update this branch just after asking for review... and please return state to needs_work before doing so.

Answer 97 · 2020-07-16T18:04:19.000Z

comment:80

Thanks for taking the time to test.

Answer 98 · 2020-07-16T19:58:35.000Z

comment:81

Debian testing running on core i7 + 16 GB RAM
Sage configured to use system's R (4.0.2, as packaged by Debian)
Rebased on 9.2.beta5 (penibly : 5 merge conflicts...)

Compiles fine. Some rough tests in the Jupyter notebook.

sage -t --long --warn-long 194.3 src/sage/interfaces/r.py
    [257 tests, 4.80 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 4.9 seconds
    cpu time: 4.5 seconds
    cumulative wall time: 4.8 seconds

ptestlong underway.

Answer 99 · 2020-07-16T20:03:40.000Z

comment:82

Replying to @EmmanuelCharpentier:

Rebased on 9.2.beta5 (penibly : 5 merge conflicts...)

Thanks for pointing this out - I've just merged 9.2.beta5 into dependency #30118

Answer 100 · 2020-07-16T20:05:37.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`5d13238`	`Merge tag '9.2.beta5' into t/30118/handle_sage_check_packages_in_build_make_makefile_in__not_sage_spkg`
`1cf262c`	`Merge branch 't/30118/handle_sage_check_packages_in_build_make_makefile_in__not_sage_spkg' into t/29441/public/29441`