Reduce R to a dummy package, upgrade rpy2 to 3.4.5
Closed this issue · 68 comments
We reduce the unmaintained r package (stuck at 3.6.3) so that we detect a system installation of R (>= 3.5) but do not attempt to install it ourselves.
When R is found as a system package, rpy2 will be built; otherwise it won't.
rpy2 3.5.x only supports R >= 4.0 according to
https://rpy2.github.io/doc/v3.5.x/html/overview.html#requirements
Here we upgrade rpy2 to 3.4.5, the latest that still supports R >= 3.5 according to https://rpy2.github.io/doc/v3.4.x/html/overview.html#requirements
Support for R < 3.5 was already dropped in the rpy 3.3.x series (our current version of rpy2 is 3.3.6).
According to https://repology.org/project/r/versions, we will thus drop support for R on the ancient platforms:
ubuntu-trusty(R 3.0.2; past End of Standard Support; EOL April 2024]),ubuntu-xenial(R 3.2.3; past End of Standard Support; EOL April 2026),ubuntu-bionic(R 3.4.4; End of Standard Support: April 2023)debian-stretch(R 3.3.3; past EOL)fedora-26(R 3.4.4; past EOL),
as well as on Cygwin. (See #32074)
Depends on #34193
CC: @EmmanuelCharpentier @videlec @dimpase @isuruf @jhpalmieri @novoselt @williamstein
Component: packages: standard
Author: Matthias Koeppe
Branch/Commit: a50460e
Reviewer: John Palmieri
Issue created by migration from https://trac.sagemath.org/ticket/34268
good idea
Description changed:
---
+++
@@ -1,3 +1,3 @@
We reduce the unmaintained `r` package (stuck at 3.6.3) so that we detect a system installation of R but do not attempt to install it ourselves.
-When found as a system package, `rpy2` will be built; otherwise it won't.
+When R is found as a system package, `rpy2` will be built; otherwise it won't.Branch pushed to git repo; I updated commit sha1. New commits:
cbcff37 | build/pkgs/rpy2: Mark as not required when there is no R |
Commit: cbcff37
Branch pushed to git repo; I updated commit sha1. New commits:
004bac5 | build/pkgs/r: Restore spkg-legacy-uninstall as spkg-uninstall script |
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
1b61561 | m4/sage_spkg_collect.m4: Display installed package version |
215fbbc | m4/sage_spkg_collect.m4: Omit 'version none' in messages |
8273456 | m4/sage_spkg_collect.m4: Show installed versions / upgrade information |
9574e8c | m4/sage_spkg_collect.m4: Handle multiple installation records better |
b1a4aac | build/make/Makefile.in: For script packages, remove old installation records |
515a785 | m4/sage_spkg_collect.m4: Update comment regarding multiple installation records of script packages |
46a0870 | m4/sage_spkg_collect.m4: Actually find installation records in SAGE_VENV when that is set implicitly |
7bda1cb | m4/sage_spkg_collect.m4: Restore uninstall message |
551b2a2 | Merge #34193 |
77a2474 | build/make/Makefile.in: When uninstalling script packages, remove all versions of stamp files |
Author: Matthias Koeppe
Branch pushed to git repo; I updated commit sha1. New commits:
e54b60c | build/pkgs/r/spkg-configure.m4: Bump required r version for rpy2 |
Description changed:
---
+++
@@ -1,3 +1,13 @@
-We reduce the unmaintained `r` package (stuck at 3.6.3) so that we detect a system installation of R but do not attempt to install it ourselves.
+We reduce the unmaintained `r` package (stuck at 3.6.3) so that we detect a system installation of R (>= 3.5) but do not attempt to install it ourselves.
When R is found as a system package, `rpy2` will be built; otherwise it won't.
+
+rpy2 3.5.x only supports R >= 4.0 according to
+https://rpy2.github.io/doc/v3.5.x/html/overview.html#requirements
+
+Here we upgrade rpy2 to 3.4.5, the latest that still supports R >= 3.5 according to https://rpy2.github.io/doc/v3.4.x/html/overview.html#requirements
+
+Support for R < 3.5 was already dropped in the rpy 3.3.x series (our current version of rpy2 is 3.3.6).
+
+According to https://repology.org/project/r/versions, we will thus drop support for R on the ancient platforms `ubuntu-trusty` (R 3.0.2), `ubuntu-xenial` (R 3.2.3), `ubuntu-bionic` (R 3.4.4), `debian-stretch` (R 3.3.3), and `fedora-26` (R 3.4.4).
+Branch pushed to git repo; I updated commit sha1. New commits:
83a751a | build/pkgs/rpy2: Downgrade to 3.4.5 |
Description changed:
---
+++
@@ -9,5 +9,5 @@
Support for R < 3.5 was already dropped in the rpy 3.3.x series (our current version of rpy2 is 3.3.6).
-According to https://repology.org/project/r/versions, we will thus drop support for R on the ancient platforms `ubuntu-trusty` (R 3.0.2), `ubuntu-xenial` (R 3.2.3), `ubuntu-bionic` (R 3.4.4), `debian-stretch` (R 3.3.3), and `fedora-26` (R 3.4.4).
+According to https://repology.org/project/r/versions, we will thus drop support for R on the ancient platforms `ubuntu-trusty` (R 3.0.2), `ubuntu-xenial` (R 3.2.3), `ubuntu-bionic` (R 3.4.4), `debian-stretch` (R 3.3.3), and `fedora-26` (R 3.4.4). (See #32074)
Description changed:
---
+++
@@ -9,5 +9,5 @@
Support for R < 3.5 was already dropped in the rpy 3.3.x series (our current version of rpy2 is 3.3.6).
-According to https://repology.org/project/r/versions, we will thus drop support for R on the ancient platforms `ubuntu-trusty` (R 3.0.2), `ubuntu-xenial` (R 3.2.3), `ubuntu-bionic` (R 3.4.4), `debian-stretch` (R 3.3.3), and `fedora-26` (R 3.4.4). (See #32074)
+According to https://repology.org/project/r/versions, we will thus drop support for R on the ancient platforms `ubuntu-trusty` (R 3.0.2), `ubuntu-xenial` (R 3.2.3), `ubuntu-bionic` (R 3.4.4), `debian-stretch` (R 3.3.3), and `fedora-26` (R 3.4.4), as well as on Cygwin. (See #32074)
How about requiring R to be at least version 4 (4.0.0 was released over two years ago)? The current R version is 4.2.1.
This would allow newer rpy2.
How do the "rpy2" optional tests in interfaces/r.py do with this?
Replying to @kcrisman:
How do the "rpy2" optional tests in interfaces/r.py do with this?
No change necessary
Replying to @dimpase:
How about requiring R to be at least version 4 (4.0.0 was released over two years ago)? The current R version is 4.2.1.
This would make R unavailable on a lot more platforms, so I don't think it would be the right choice.
See https://repology.org/project/r/versions: debian-buster, fedora <= 32, opensuse <= 15.3, ubuntu <= focal
Fedora <= 32 is past EOL. We should not be concerned about it.
Further, accoding to https://cran.r-project.org/bin/linux/ubuntu/fullREADME.html, Ubunty Focal LTS has R 4.2.
Replying to @dimpase:
Further, accoding to https://cran.r-project.org/bin/linux/ubuntu/fullREADME.html, Ubunty Focal LTS has R 4.2.
No, that's not what the page says.
They are offering a custom repository with their own package for ubuntu focal.
Other than just the fact that it's newer, what is the compelling reason to push for a later version than 3.4.5 of rpy2?
I don't think there's a good reason, in particular given that there is zero evidence for the existence of any active users of our R interface
Replying to @mkoeppe:
I don't think there's a good reason, in particular given that there is zero evidence for the existence of any active users of our R interface
Just a few hours ago I got a request for some improvements to how R is handled in SageMathCell, which does rely on our R interface. There were other discussions involving it over the years. So while I cannot say how wide-spread it is, there are certainly users out there and I would appreciate continuation of the support which is available via standard means, i.e. either by installing Sage and its packages, or using standard system packages. Using other custom repositories is highly undesirable as it tends to break over time.
Are you using system R on SageMathCell or a build of R from the SPKG?
SPKG, as far as I know. I didn't explore if it is already an option to use the system one.
I don't think there's a good reason, in particular given that there is zero evidence for the existence of any active users of our R interface
Just a few hours ago I got a request for some improvements to how R is handled in SageMathCell, which does rely on our R interface. There were other discussions involving it over the years. So while I cannot say how wide-spread it is, there are certainly users out there
Thanks for pointing this out. R (via the zillions of packages) provides access to so many weird numerical methods and other things that removing the interface seems unwise, even if it's not a prime use of Sage. (This conversation seems to come up about every three years on sage-devel as well, where people come out of the woodwork; I've used the R interface for research purposes on several occasions myself.)
Replying to @kcrisman:
removing the interface seems unwise
This is not proposed on this ticket
removing the interface seems unwise
This is not proposed on this ticket
no worries, just being proactive regarding the phrase "given that there is zero evidence" which could be misinterpreted
Replying to @novoselt:
SPKG, as far as I know. I didn't explore if it is already an option to use the system one.
The files in https://github.com/sagemath/sagecell/tree/master/contrib/vm suggest that you are using ubuntu-bionic, so if this is what you use in production, you will be affected by the proposed change.
The ticket is marked for the Sage 9.8 milestone, so you'll have plenty of time to do an upgrade to the current LTS (ubuntu-jammy)
Description changed:
---
+++
@@ -9,5 +9,11 @@
Support for R < 3.5 was already dropped in the rpy 3.3.x series (our current version of rpy2 is 3.3.6).
-According to https://repology.org/project/r/versions, we will thus drop support for R on the ancient platforms `ubuntu-trusty` (R 3.0.2), `ubuntu-xenial` (R 3.2.3), `ubuntu-bionic` (R 3.4.4), `debian-stretch` (R 3.3.3), and `fedora-26` (R 3.4.4), as well as on Cygwin. (See #32074)
+According to https://repology.org/project/r/versions, we will thus drop support for R on the ancient platforms:
+- `ubuntu-trusty` (R 3.0.2),
+- `ubuntu-xenial` (R 3.2.3),
+- `ubuntu-bionic` (R 3.4.4; End of Standard Support: April 2023)
+- `debian-stretch` (R 3.3.3)
+- `fedora-26` (R 3.4.4),
+as well as on Cygwin. (See #32074)
Description changed:
---
+++
@@ -10,10 +10,10 @@
Support for R < 3.5 was already dropped in the rpy 3.3.x series (our current version of rpy2 is 3.3.6).
According to https://repology.org/project/r/versions, we will thus drop support for R on the ancient platforms:
-- `ubuntu-trusty` (R 3.0.2),
-- `ubuntu-xenial` (R 3.2.3),
+- `ubuntu-trusty` (R 3.0.2; past [[End of Standard Support](https://wiki.ubuntu.com/Releases); EOL April 2024]),
+- `ubuntu-xenial` (R 3.2.3; past End of Standard Support; EOL April 2026),
- `ubuntu-bionic` (R 3.4.4; End of Standard Support: April 2023)
-- `debian-stretch` (R 3.3.3)
-- `fedora-26` (R 3.4.4),
+- `debian-stretch` (R 3.3.3; past EOL)
+- `fedora-26` (R 3.4.4; past EOL),
as well as on Cygwin. (See #32074)
Description changed:
---
+++
@@ -10,7 +10,7 @@
Support for R < 3.5 was already dropped in the rpy 3.3.x series (our current version of rpy2 is 3.3.6).
According to https://repology.org/project/r/versions, we will thus drop support for R on the ancient platforms:
-- `ubuntu-trusty` (R 3.0.2; past [[End of Standard Support](https://wiki.ubuntu.com/Releases); EOL April 2024]),
+- `ubuntu-trusty` (R 3.0.2; past [End of Standard Support](https://wiki.ubuntu.com/Releases); EOL April 2024]),
- `ubuntu-xenial` (R 3.2.3; past End of Standard Support; EOL April 2026),
- `ubuntu-bionic` (R 3.4.4; End of Standard Support: April 2023)
- `debian-stretch` (R 3.3.3; past EOL)Replying to @jhpalmieri:
Other than just the fact that it's newer, what is the compelling reason to push for a later version than 3.4.5 of
rpy2?
Upstreams hate to support old versions, that's the main reason.
I'd say that's not concrete enough to dictate that an upgrade has to go all the way to the most recent version. (Previous updates to rpy2 were done in 2020.)
Something is going wrong with prerequisites. From sagemath_doc_html-none.log:
pkg_resources.DistributionNotFound: The 'rpy2<3.4,>=3.3' distribution was not found and is required by sagemath-standard
(The rpy2 package has not yet been built — it's still in progress trying to download the tarball. Since the downloading timeouts can take a while, is there any way to tell Sage to skip the standard mirrors and just jump straight to the option given by --enable-download-from-upstream-url?)
Replying to @jhpalmieri:
Since the downloading timeouts can take a while, is there any way to tell Sage to skip the standard mirrors and just jump straight to the option given by
--enable-download-from-upstream-url?)
You can edit the file upstream/mirror_list to remove most mirrors
Replying to @jhpalmieri:
Something is going wrong with prerequisites. From
sagemath_doc_html-none.log:pkg_resources.DistributionNotFound: The 'rpy2<3.4,>=3.3' distribution was not found and is required by sagemath-standard
Is this giving an error, or does it continue after that?
Replying to @mkoeppe:
Replying to @jhpalmieri:
Something is going wrong with prerequisites. From
sagemath_doc_html-none.log:pkg_resources.DistributionNotFound: The 'rpy2<3.4,>=3.3' distribution was not found and is required by sagemath-standardIs this giving an error, or does it continue after that?
The message repeats many times, including a traceback, but the build continues.
Right, and of course it doesn't matter whether rpy2 has been built, the same message appears, because this branch installs version 3.4.5 of rpy2, and that does not satisfy rpy2<3.4,>=3.3.
Branch pushed to git repo; I updated commit sha1. New commits:
7ab9370 | build/pkgs/rpy2/install-requires.txt: Remove upper version bound |
./configure says
notice: the following SPKGs did not find equivalent system packages:
rpy2 4ti2 _recommended cbc coxeter3 ffmpeg gp2c igraph libsemigroups lrslib pandoc pdf2svg perl_mongodb perl_term_readline_gnu polymake
Do you know why there seems to be extra space after rpy2?
With ./configure --with-system-r=no, rpy2 is not built and I see lots of warnings in docbuilding:
pkg_resources.DistributionNotFound: The 'rpy2>=3.3' distribution was not found and is required by sagemath-standard
The scipy log file says
sagemath-standard 9.7b8 requires networkx<3.0,>=2.4, which is not installed.
sagemath-standard 9.7b8 requires rpy2>=3.3, which is not installed.
(networkx-2.8.4 had not yet finished installing at this point.)
Replying to @jhpalmieri:
./configuresaysnotice: the following SPKGs did not find equivalent system packages: rpy2 4ti2 _recommended cbc coxeter3 ffmpeg gp2c igraph libsemigroups lrslib pandoc pdf2svg perl_mongodb perl_term_readline_gnu polymakeDo you know why there seems to be extra space after
rpy2?
I think I put this space there to separate standard packages from optional packages
Branch pushed to git repo; I updated commit sha1. New commits:
5d66432 | src/setup.cfg.m4: Move rpy2 from install-requires to extras-require |
Looks okay to me. Passes tests with and without using the system installation of R.
Reviewer: John Palmieri
Thank you!
Replying to @mkoeppe:
Replying to @jhpalmieri:
Since the downloading timeouts can take a while, is there any way to tell Sage to skip the standard mirrors and just jump straight to the option given by
--enable-download-from-upstream-url?)You can edit the file
upstream/mirror_listto remove most mirrors
Would it be useful to be able to add $UPSTREAM (or something like that) to the mirror list, which would have the effect of turning on --enable-download-from-upstream-url?
I think it would make sense to limit the number of mirrors it tries to some small number (5?) before it tries the upstream.
Merge failure on top of:
024b81f Trac #33812: Refactor distributions sagemath-{objects,categories} through sagemath-{environment,repl}
e4d3004 Trac #33805: use Homebrew's primecount
5076cd5 Trac #34110: numpy 1.23.x
025fdd4 Trac #34532: modernize super in manifolds/
f9129b1 Trac #34531: some details in LS paths crystals
7f7088d Trac #34529: remove deprecated method spring_layout_fast_split from src/sage/graphs/generic_graph_pyx.pyx
12890b6 Trac #34520: Remove randomly failing doctest in src/sage/graphs/generators/random.py
c8511ab Trac #34512: compute list CRT via tree
3f4e544 Trac #34508: Improve documentation of JoinFeature
cb5e1af Trac #34493: Make TrivialFamily.map return a TrivialFamily
c036ccc Trac #34486: VectorFieldModule: Faster fast path for tensor_module, exterior_power, dual_exterior_power
89b4155 Trac #34469: modern super in matrix_space
4f3a9b0 Trac #34468: fix W605 in all pyx files inside matroids/
bfa6342 Trac #34465: Invalid escape sequence in special.py
cd65d44 Trac #34462: Uniformize the headline: finite rings
7736a49 Trac #34435: Add method to trim trailing zeros from IntegerVector
154baa5 Trac #34432: Upgrade: jupyter-packaging 0.12.3
bfc2ccf Trac #34409: remove experimental warning for composite elliptic-curve isogenies
89b0d28 Trac #34375: Replace sage.algebras.yangian.GeneratorIndexingSet with cartesian_product
ce82f81 Trac #34358: pycodestyle cleanup in src/sage/graphs/generic_graph.py (part 4)
c8c541f Trac #34578: standard error messages in pbori
721c4c1 Trac #34573: Remove binary_function from ginac's code in sage
5fc16f5 Trac #34571: fix typo in docstring
b836c6c Trac #34562: refactor multiple zeta values
1366a0d Trac #34528: Installation guide: Improve section on prerequisites/recommendations
ad97e1b Trac #34413: implement derivatives of lazy series
a802d07 Trac #31664: Add package msolve 0.4.4 (multivariate polynomial system solver)
4541564 Updated SageMath version to 9.8.beta0
merge was not clean: conflicts in build/make/Makefile.in
Changed branch from u/mkoeppe/reduce_r_to_a_dummy_package__upgrade_rpy2 to a50460e