Restore "huge" package type
mkoeppe opened this issue · 28 comments
(from #29901)
... for packages like database_stein_watkins and polytopes_db_4d
Parts of sage_bootstrap know about this package type, but other scripts don't.
Our GH Actions scripts filter out these packages by name. This can be refactored.
With the present ticket, we can now say sage --package list :optional: :experimental: --no-file huge to filter out the huge packages.
Huge packages are now marked by the presence of the file huge; so this is actually orthogonal to the package type.
We also add files named has_nonfree_dependencies to some other packages so that they can be filtered out by the same mechanism.
CC: @jhpalmieri @dimpase @orlitzky @seblabbe @Etn40ff @tobiasdiez
Component: build
Author: Matthias Koeppe
Branch/Commit: 15cc20c
Reviewer: John Palmieri
Issue created by migration from https://trac.sagemath.org/ticket/29919
Description changed:
---
+++
@@ -1,3 +1,7 @@
(from #29901)
... for packages like `database_stein_watkins` and `polytopes_db_4d`
+
+Parts of `sage_bootstrap` know about this package type, but other scripts don't.
+
+Setting new milestone based on a cursory review of ticket status, priority, and last modification date.
Description changed:
---
+++
@@ -4,4 +4,5 @@
Parts of `sage_bootstrap` know about this package type, but other scripts don't.
+Our GH Actions scripts filter out these packages by name. This can be refactored.
Author: Matthias Koeppe
Description changed:
---
+++
@@ -6,3 +6,5 @@
Our GH Actions scripts filter out these packages by name. This can be refactored.
+With the present ticket, we can now say `sage --package list :optional: :experimental: --no-file huge` to filter out the huge packages.
+Description changed:
---
+++
@@ -8,3 +8,8 @@
With the present ticket, we can now say `sage --package list :optional: :experimental: --no-file huge` to filter out the huge packages.
+Huge packages are now marked by the presence of the file `huge`; so this is actually orthogonal to the package type.
+
+We also add files named `has_nonfree_dependencies` to some other packages so that they can be filtered out by the same mechanism.
+
+Branch pushed to git repo; I updated commit sha1. New commits:
554ad59 | build/pkgs/{scipoptsuite,sage_numerical_backends_cplex,sage_numerical_backends_gurobi}/has_nonfree_dependencies: New |
Can you document both huge and has_nonfree_dependencies in the developer's guide?
Good idea
I think we should have a cleaner way to add "tags" to packages, but in lieu of general solution this gets the job done.
The nonfree-dependencies thing is a bit of a legal mess. Users who install such a package lose the right to distribute their copy of sage. Entities such as universities may not want to wade into that sort of grey area. Anyway, that's orthogonal to this ticket.
Branch pushed to git repo; I updated commit sha1. New commits:
b401649 | src/doc/en/developer/packaging.rst: Explain package tags has_nonfree_dependencies, huge |
Branch pushed to git repo; I updated commit sha1. New commits:
bf0c923 | src/doc/en/developer/packaging.rst: Add section title |
Branch pushed to git repo; I updated commit sha1. New commits:
15cc20c | src/doc/en/developer/packaging.rst: Typo |
Would
if any(not pkg.has_file(filename) for filename in filenames):
be more efficient than the current
if not all(pkg.has_file(filename) for filename in filenames):
and similarly a few lines later with not any(...)? Or does Python do this sort of optimization automatically?
Never mind, I guess it doesn't matter.
There's no difference in efficiency - both any and all shortcircuit.
And I think there is no difference in clarity
I got confused and thought that the two might short-circuit at different points, but they will short-circuit at exactly the same point, so it makes no difference.
This is not your fault and should not be fixed on this ticket, but I wish that the code in expand_class.py were documented. Anyway, let's move forward with this.
Reviewer: John Palmieri
Thanks!
Changed branch from u/mkoeppe/restore__huge__package_type to 15cc20c