Install src/bin scripts by sagelib's setup.py, not make

Question

Install src/bin scripts by sagelib's setup.py, not make

Closed this issue 5 years ago · 83 comments

This ticket cleans up how the contents of src/bin are installed. One of its goals is to make the sagelib installation done by src/setup.py self-contained regarding the scripts that it needs at runtime.

Previous tickets moved some of the scripts to build/bin (no longer installed in SAGE_LOCAL).

In follow-up tickets, src/bin could be cleaned up more.

See the list of files added on this ticket to src/setup.py for beginnings of a categorization.
#29850: Install src/bin/sage-env-config as part of sage_conf
#29847: Install sage_setup with setuptools as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"

Depends on #29111

CC: @jdemeyer @vbraun @embray @isuruf @jhpalmieri @kiwifb @kcrisman @orlitzky @antonio-rojas @dimpase

Component: build

Author: Matthias Koeppe

Branch/Commit: 19d7f67

Reviewer: John Palmieri, François Bissey

Issue created by migration from https://trac.sagemath.org/ticket/21559

Answer 1 · 2016-09-27T21:50:35.000Z

Description changed:

--- 
+++ 
@@ -1,5 +1,6 @@
 This list includes probably:
 - `sage-download-upstream`
 - `sage-fix-pkg-checksums`
+- `testcc.sh` etc. (which are used by some spkg without a proper build system)
 and perhaps more.

Answer 2 · 2016-09-28T06:23:48.000Z

comment:2

+1

Also sage-pip-install of #21441, but I might wait until that ticket is merged first, then move that script along with the others.

Answer 3 · 2016-09-29T14:49:27.000Z

comment:3

Replying to @embray:

+1

Also sage-pip-install of #21441

I already moved that in #21441 (which still needs review by the way).

Answer 4 · 2016-10-15T06:09:26.000Z

Description changed:

--- 
+++ 
@@ -4,3 +4,4 @@
 - `testcc.sh` etc. (which are used by some spkg without a proper build system)
 and perhaps more.
 
+See also #21707: Split sage-env into sage-build-env and sage-env

Answer 5 · 2018-06-02T04:21:28.000Z

Description changed:

--- 
+++ 
@@ -2,6 +2,8 @@
 - `sage-download-upstream`
 - `sage-fix-pkg-checksums`
 - `testcc.sh` etc. (which are used by some spkg without a proper build system)
+- #25130: `sage-dist-helpers`
+
 and perhaps more.
 
 See also #21707: Split sage-env into sage-build-env and sage-env

Answer 6 · 2018-08-03T19:20:18.000Z

comment:6

update milestone 8.3 -> 8.4

Answer 7 · 2019-05-20T00:41:02.000Z

Description changed:

--- 
+++ 
@@ -3,6 +3,7 @@
 - `sage-fix-pkg-checksums`
 - `testcc.sh` etc. (which are used by some spkg without a proper build system)
 - #25130: `sage-dist-helpers`
+- #23769: `src/bin/sage-starts` should be moved to `build/bin` or merged into `build/make/deps`
 
 and perhaps more.

Answer 8 · 2019-06-14T14:54:19.000Z

comment:8

As the Sage-8.8 release milestone is pending, we should delete the sage-8.8 milestone for tickets that are not actively being worked on or that still require significant work to move forward. If you feel that this ticket should be included in the next Sage release at the soonest please set its milestone to the next release milestone (sage-8.9).

Answer 9 · 2020-02-21T19:14:49.000Z

Description changed:

--- 
+++ 
@@ -7,4 +7,72 @@
 
 and perhaps more.
 
+Full list of `src/bin` as of 9.1.beta4:
+
+```
+math-readline
+sage
+sage-README-osx.txt
+sage-cachegrind
+sage-callgrind
+sage-cleaner
+sage-clone-source
+sage-coverage
+sage-coverageall
+sage-cython
+sage-download-upstream
+sage-env
+sage-env-config
+sage-env-config.in
+sage-eval
+sage-fix-pkg-checksums
+sage-fixdoctests
+sage-gdb-commands
+sage-grep
+sage-grepdoc
+sage-inline-fortran
+sage-ipynb2rst
+sage-ipython
+sage-list-experimental
+sage-list-optional
+sage-list-packages
+sage-list-standard
+sage-location
+sage-massif
+sage-maxima.lisp
+sage-native-execute
+sage-notebook
+sage-num-threads.py
+sage-omega
+sage-open
+sage-pkg
+sage-preparse
+sage-pypkg-location
+sage-python
+sage-rebase.bat
+sage-rebase.sh
+sage-rebaseall.bat
+sage-rebaseall.sh
+sage-rst2sws
+sage-rst2txt
+sage-rsyncdist
+sage-run
+sage-run-cython
+sage-runtests
+sage-sdist
+sage-starts
+sage-startuptime.py
+sage-sws2rst
+sage-unzip
+sage-update-src
+sage-update-version
+sage-upgrade
+sage-valgrind
+sage-version.sh
+testcc.sh
+testcflags.sh
+testcxx.sh
+```
+
+
 See also #21707: Split sage-env into sage-build-env and sage-env

Answer 10 · 2020-02-21T19:35:04.000Z

Description changed:

--- 
+++ 
@@ -9,12 +9,37 @@
 
 Full list of `src/bin` as of 9.1.beta4:
 
+Move to `build/bin` (no longer installed in SAGE_LOCAL):
+
+```
+# Used only by some build/pkgs/SPKG/spkg-install scripts, not by anything in src
+testcc.sh
+testcflags.sh
+testcxx.sh
+```
+
+To stay in `src/bin`, install by `src/setup.py` (#21569):
+
+```
+sage
+# Helper scripts invoked by sage script
+sage-cachegrind
+sage-callgrind
+sage-valgrind
+
+```
+
+Not an executable script, move elsewhere:
+
+```
+sage-README-osx.txt      # ???
+
+```
+
+To be determined:
+
 ```
 math-readline
-sage
-sage-README-osx.txt
-sage-cachegrind
-sage-callgrind
 sage-cleaner
 sage-clone-source
 sage-coverage
@@ -67,11 +92,7 @@
 sage-update-src
 sage-update-version
 sage-upgrade
-sage-valgrind
 sage-version.sh
-testcc.sh
-testcflags.sh
-testcxx.sh
 ```

Answer 11 · 2020-02-22T22:47:32.000Z

Description changed:

--- 
+++ 
@@ -29,10 +29,16 @@
 
 ```
 
-Not an executable script, move elsewhere:
+Obsolete, delete:
 
 ```
-sage-README-osx.txt      # ???
+sage-README-osx.txt      # see #25995, #21570
+
+```
+
+Not an executable script, move elsewhere (#21570):
+
+```
 
 ```

Answer 12 · 2020-03-02T23:45:06.000Z

Branch: u/mkoeppe/change-src-bin-installation

Answer 13 · 2020-03-03T00:00:14.000Z

Description changed:

--- 
+++ 
@@ -1,13 +1,4 @@
-This list includes probably:
-- `sage-download-upstream`
-- `sage-fix-pkg-checksums`
-- `testcc.sh` etc. (which are used by some spkg without a proper build system)
-- #25130: `sage-dist-helpers`
-- #23769: `src/bin/sage-starts` should be moved to `build/bin` or merged into `build/make/deps`
-
-and perhaps more.
-
-Full list of `src/bin` as of 9.1.beta4:
+This ticket cleans up `src/bin` and how its contents are installed.
 
 Move to `build/bin` (no longer installed in SAGE_LOCAL):
 
@@ -18,7 +9,13 @@
 testcxx.sh
 ```
 
-To stay in `src/bin`, install by `src/setup.py` (#21569):
+Obsolete, delete:
+
+```
+sage-README-osx.txt      # see #25995, #21570
+```
+
+To stay in `src/bin`, install by `src/setup.py` using the `scripts` feature, instead of by `build/make/Makefile` (after scanning them in `configure`) (this used to be ticket #21569):
 
 ```
 sage
@@ -26,15 +23,9 @@
 sage-cachegrind
 sage-callgrind
 sage-valgrind
-
+...
 ```
 
-Obsolete, delete:
-
-```
-sage-README-osx.txt      # see #25995, #21570
-
-```
 
 Not an executable script, move elsewhere (#21570):
 
@@ -42,7 +33,7 @@
 
 ```
 
-To be determined:
+To be determined (see list added to `src/setup.py`):
 
 ```
 math-readline
@@ -51,12 +42,12 @@
 sage-coverage
 sage-coverageall
 sage-cython
-sage-download-upstream
+sage-download-upstream                 # -> build?
 sage-env
 sage-env-config
 sage-env-config.in
 sage-eval
-sage-fix-pkg-checksums
+sage-fix-pkg-checksums               # -> build?
 sage-fixdoctests
 sage-gdb-commands
 sage-grep
@@ -91,7 +82,7 @@
 sage-run-cython
 sage-runtests
 sage-sdist
-sage-starts
+sage-starts                    #23769: `src/bin/sage-starts` should be moved to `build/bin` or merged into `build/make/deps`
 sage-startuptime.py
 sage-sws2rst
 sage-unzip
@@ -101,5 +92,6 @@
 sage-version.sh
 ```
 
+(Based on the full list of `src/bin` as of 9.1.beta4.)
 
 See also #21707: Split sage-env into sage-build-env and sage-env

Answer 14 · 2020-03-03T00:00:14.000Z

Commit: 428ea11

Answer 15 · 2020-03-03T00:00:14.000Z

Author: Matthias Koeppe

Answer 16 · 2020-03-03T00:00:14.000Z

New commits:

`1a54a80`	`Move src/bin/testc*.sh to build/bin/, delete src/bin/sage-README-osx.txt`
`428ea11`	`Install src/bin/* by setup.py [scripts] instead of by build/make/deps`

Answer 17 · 2020-03-03T00:17:30.000Z

Changed commit from 428ea11 to 1b3c860

Answer 18 · 2020-03-03T00:17:30.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`1b3c860`	`Move src/bin/sage-starts to build/bin/`

Answer 19 · 2020-03-03T00:21:26.000Z

Description changed:

--- 
+++ 
@@ -7,6 +7,8 @@
 testcc.sh
 testcflags.sh
 testcxx.sh
+# Only used by build/make/deps
+sage-starts                    
 ```
 
 Obsolete, delete:
@@ -15,17 +17,9 @@
 sage-README-osx.txt      # see #25995, #21570
 ```
 
-To stay in `src/bin`, install by `src/setup.py` using the `scripts` feature, instead of by `build/make/Makefile` (after scanning them in `configure`) (this used to be ticket #21569):
+This ticket leaves all remaining files in `src/bin`, but changes their installation to be done by `src/setup.py` using the `scripts` feature, instead of by `build/make/Makefile` (after scanning them in `configure`) (this used to be ticket #21569).
 
-```
-sage
-# Helper scripts invoked by sage script
-sage-cachegrind
-sage-callgrind
-sage-valgrind
-...
-```
-
+In follow-up tickets, `src/bin` could be cleaned up more.  See the list of files added on this ticket to `src/setup.py`.
 
 Not an executable script, move elsewhere (#21570):
 
@@ -33,7 +27,7 @@
 
 ```
 
-To be determined (see list added to `src/setup.py`):
+To be determined (see list):
 
 ```
 math-readline
@@ -82,7 +76,6 @@
 sage-run-cython
 sage-runtests
 sage-sdist
-sage-starts                    #23769: `src/bin/sage-starts` should be moved to `build/bin` or merged into `build/make/deps`
 sage-startuptime.py
 sage-sws2rst
 sage-unzip
@@ -94,4 +87,5 @@
 
 (Based on the full list of `src/bin` as of 9.1.beta4.)
 
-See also #21707: Split sage-env into sage-build-env and sage-env
+See also 
+- #21707: Split sage-env into sage-build-env and sage-env

Answer 20 · 2020-03-03T00:22:30.000Z

Description changed:

--- 
+++ 
@@ -54,12 +54,10 @@
 sage-list-packages
 sage-list-standard
 sage-location
-sage-massif
 sage-maxima.lisp
 sage-native-execute
 sage-notebook
 sage-num-threads.py
-sage-omega
 sage-open
 sage-pkg
 sage-preparse

Answer 21 · 2020-03-08T20:38:09.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`546d1b0`	`Merge tag '9.1.beta7' into t/21559/change-src-bin-installation`

Answer 22 · 2020-03-08T20:38:09.000Z

Changed commit from 1b3c860 to 546d1b0

Answer 23 · 2020-03-17T03:45:24.000Z

comment:22

This ticket could use some help from distribution packagers...

Answer 24 · 2020-03-18T01:56:19.000Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`2850b50`	`Move src/bin/testc*.sh to build/bin/, delete src/bin/sage-README-osx.txt`
`4466fd6`	`Install src/bin/* by setup.py [scripts] instead of by build/make/deps`
`26b7596`	`Move src/bin/sage-starts to build/bin/`
`dddd462`	`Remove obsolete script sage-pkg (creates old-style spkg archives)`

Answer 25 · 2020-03-18T01:56:19.000Z

Changed commit from 546d1b0 to dddd462

Answer 26 · 2020-03-18T02:02:02.000Z

Changed commit from dddd462 to 9825799

Answer 27 · 2020-03-18T02:02:02.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`9825799`	`src/bin/sage-unzip: Remove (unused script)`

Answer 28 · 2020-03-18T02:28:30.000Z

Description changed:

--- 
+++ 
@@ -15,6 +15,8 @@
 
 ```
 sage-README-osx.txt      # see #25995, #21570
+sage-pkg       # a script to create old-style spkgs
+sage-unzip   # unused
 ```
 
 This ticket leaves all remaining files in `src/bin`, but changes their installation to be done by `src/setup.py` using the `scripts` feature, instead of by `build/make/Makefile` (after scanning them in `configure`) (this used to be ticket #21569).
@@ -59,7 +61,6 @@
 sage-notebook
 sage-num-threads.py
 sage-open
-sage-pkg
 sage-preparse
 sage-pypkg-location
 sage-python
@@ -76,7 +77,6 @@
 sage-sdist
 sage-startuptime.py
 sage-sws2rst
-sage-unzip
 sage-update-src
 sage-update-version
 sage-upgrade

Answer 29 · 2020-03-20T20:46:55.000Z

Dependencies: #23769, #29381, #29383

Answer 30 · 2020-03-21T13:41:48.000Z

Description changed:

--- 
+++ 
@@ -17,6 +17,15 @@
 sage-README-osx.txt      # see #25995, #21570
 sage-pkg       # a script to create old-style spkgs
 sage-unzip   # unused
+```
+
+Install by a script package `sage-distribution-scripts`? These would be scripts that only sage-the-distribution would install in SAGE_LOCAL, but would not be in OS packages:
+
+```
+sage-list-experimental
+sage-list-optional
+sage-list-packages
+sage-list-standard
 ```
 
 This ticket leaves all remaining files in `src/bin`, but changes their installation to be done by `src/setup.py` using the `scripts` feature, instead of by `build/make/Makefile` (after scanning them in `configure`) (this used to be ticket #21569).
@@ -51,10 +60,6 @@
 sage-inline-fortran
 sage-ipynb2rst
 sage-ipython
-sage-list-experimental
-sage-list-optional
-sage-list-packages
-sage-list-standard
 sage-location
 sage-maxima.lisp
 sage-native-execute

Answer 31 · 2020-03-21T13:47:36.000Z

Description changed:

--- 
+++ 
@@ -28,7 +28,20 @@
 sage-list-standard
 ```
 
-This ticket leaves all remaining files in `src/bin`, but changes their installation to be done by `src/setup.py` using the `scripts` feature, instead of by `build/make/Makefile` (after scanning them in `configure`) (this used to be ticket #21569).
+Left in `src/bin`, installation changed to be done by `src/setup.py` using the `scripts` feature, instead of by `build/make/Makefile` (after scanning them in `configure`) (this used to be ticket #21569). All of these scripts must be useful in OS packaging of sagelib as well:
+
+```
+sage
+sage-eval       #??
+
+```
+
+Helper scripts that have no business of being in the user's PATH. They belong to something like libexec. Perhaps `src/sage/ext_data/libexec`?
+
+```
+sage-version.sh
+
+```
 
 In follow-up tickets, `src/bin` could be cleaned up more.  See the list of files added on this ticket to `src/setup.py`.
 
@@ -51,7 +64,6 @@
 sage-env
 sage-env-config
 sage-env-config.in
-sage-eval
 sage-fix-pkg-checksums               # -> build?
 sage-fixdoctests
 sage-gdb-commands
@@ -85,7 +97,6 @@
 sage-update-src
 sage-update-version
 sage-upgrade
-sage-version.sh
 ```
 
 (Based on the full list of `src/bin` as of 9.1.beta4.)

Answer 32 · 2020-03-21T15:41:32.000Z

Description changed:

--- 
+++ 
@@ -1,4 +1,4 @@
-This ticket cleans up `src/bin` and how its contents are installed.
+This ticket cleans up `src/bin` and how its contents are installed. One of its goals is to make the `sagelib` installation done by `src/setup.py` self-contained regarding the scripts that it needs at runtime.
 
 Move to `build/bin` (no longer installed in SAGE_LOCAL):

Answer 33 · 2020-03-21T18:40:52.000Z

comment:31

This breaks if you try to install sagelib with setup.py without running configure first (as some distros do) since sage-env-config doesn't exist in that case. At least sage-env and sage-env-config should be optional.

Answer 34 · 2020-03-21T20:41:39.000Z

comment:32

Can this build also configure a better link or communication between SAGE_LOCAL or SAGE_SCRIPTS_DIR variables?

Answer 35 · 2020-03-22T03:21:19.000Z

comment:33

Replying to @Shlokatadistance:

Can this build also configure a better link or communication between SAGE_LOCAL or SAGE_SCRIPTS_DIR variables?

Take a look at #25486

Answer 36 · 2020-06-12T21:14:36.000Z

Changed dependencies from #23769, #29381, #29383 to #29702

Answer 37 · 2020-06-12T21:29:22.000Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

`5db5318`	`Trac #29345: remove "break" statements from AC_SEARCH_LIBS.`
`e810ad1`	`Trac #29345: don't use sage's config.status for the lrcalc build.`
`93c9921`	`Trac #29345: replace the function that populates the CVXOPT_* variables.`
`0e66a0a`	`Trac #29345: add Dima's SPKG patches for ksh compatibility.`
`df3f05e`	`build/make/Makefile.in [SCRIPT_PACKAGE_templ]: cd into the SPKG directory; adjust spkg-install scripts`
`5372065`	`Merge branch 't/29793/script_packages_should_cd_into_the_spkg_directory' into t/29411/make_sagelib_a_script_package`
`c166b97`	`Merge branch 't/29411/make_sagelib_a_script_package' into t/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup`
`cc30471`	`build/bin/write-dockerfile.sh: Do not ADD removed file src/Makefile.in`
`8a41326`	`Merge branch 't/29411/make_sagelib_a_script_package' into t/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup`
`7c8a030`	`Install src/bin/* by setup.py [scripts] instead of by build/make/Makefile.in`

Answer 38 · 2020-06-12T21:29:22.000Z

Changed commit from 9825799 to 7c8a030

Answer 39 · 2020-06-12T21:37:03.000Z

Description changed:

--- 
+++ 
@@ -1,105 +1,5 @@
-This ticket cleans up `src/bin` and how its contents are installed. One of its goals is to make the `sagelib` installation done by `src/setup.py` self-contained regarding the scripts that it needs at runtime.
+This ticket cleans up how the contents of `src/bin` are installed. One of its goals is to make the `sagelib` installation done by `src/setup.py` self-contained regarding the scripts that it needs at runtime.
 
-Move to `build/bin` (no longer installed in SAGE_LOCAL):
+Previous tickets moved some of the scripts to `build/bin` (no longer installed in `SAGE_LOCAL`). In follow-up tickets, `src/bin` could be cleaned up more.  See the list of files added on this ticket to `src/setup.py` for beginnings of a categorization.
+ 
 
-```
-# Used only by some build/pkgs/SPKG/spkg-install scripts, not by anything in src
-testcc.sh
-testcflags.sh
-testcxx.sh
-# Only used by build/make/deps
-sage-starts                    
-```
-
-Obsolete, delete:
-
-```
-sage-README-osx.txt      # see #25995, #21570
-sage-pkg       # a script to create old-style spkgs
-sage-unzip   # unused
-```
-
-Install by a script package `sage-distribution-scripts`? These would be scripts that only sage-the-distribution would install in SAGE_LOCAL, but would not be in OS packages:
-
-```
-sage-list-experimental
-sage-list-optional
-sage-list-packages
-sage-list-standard
-```
-
-Left in `src/bin`, installation changed to be done by `src/setup.py` using the `scripts` feature, instead of by `build/make/Makefile` (after scanning them in `configure`) (this used to be ticket #21569). All of these scripts must be useful in OS packaging of sagelib as well:
-
-```
-sage
-sage-eval       #??
-
-```
-
-Helper scripts that have no business of being in the user's PATH. They belong to something like libexec. Perhaps `src/sage/ext_data/libexec`?
-
-```
-sage-version.sh
-
-```
-
-In follow-up tickets, `src/bin` could be cleaned up more.  See the list of files added on this ticket to `src/setup.py`.
-
-Not an executable script, move elsewhere (#21570):
-
-```
-
-```
-
-To be determined (see list):
-
-```
-math-readline
-sage-cleaner
-sage-clone-source
-sage-coverage
-sage-coverageall
-sage-cython
-sage-download-upstream                 # -> build?
-sage-env
-sage-env-config
-sage-env-config.in
-sage-fix-pkg-checksums               # -> build?
-sage-fixdoctests
-sage-gdb-commands
-sage-grep
-sage-grepdoc
-sage-inline-fortran
-sage-ipynb2rst
-sage-ipython
-sage-location
-sage-maxima.lisp
-sage-native-execute
-sage-notebook
-sage-num-threads.py
-sage-open
-sage-preparse
-sage-pypkg-location
-sage-python
-sage-rebase.bat
-sage-rebase.sh
-sage-rebaseall.bat
-sage-rebaseall.sh
-sage-rst2sws
-sage-rst2txt
-sage-rsyncdist
-sage-run
-sage-run-cython
-sage-runtests
-sage-sdist
-sage-startuptime.py
-sage-sws2rst
-sage-update-src
-sage-update-version
-sage-upgrade
-```
-
-(Based on the full list of `src/bin` as of 9.1.beta4.)
-
-See also 
-- #21707: Split sage-env into sage-build-env and sage-env

Answer 40 · 2020-06-12T21:38:50.000Z

Changed commit from 7c8a030 to 64ec6ae

Answer 41 · 2020-06-12T21:38:50.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`64ec6ae`	`src/setup.py [scripts]: Categorize scripts`

Answer 42 · 2020-06-13T00:38:54.000Z

Changed commit from 64ec6ae to 1ee054c

Answer 43 · 2020-06-13T00:38:54.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`1ee054c`	`Fixup`

Answer 44 · 2020-06-18T04:55:25.000Z

Changed commit from 1ee054c to daa045c

Answer 45 · 2020-06-18T04:55:25.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`daa045c`	`src/setup.py: Do not install deprecated script bin/sage-fix-pkg-checksums`

Answer 46 · 2020-06-18T05:56:19.000Z

comment:42

(this script is deleted in #29111)

Answer 47 · 2020-06-20T23:38:41.000Z

Changed dependencies from #29702 to #29702, #29111, #29920

Answer 48 · 2020-06-20T23:40:19.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`e3d9cf2`	`src/setup.py: Do not install deprecated scripts sage-list-{experimental,optional,standard}`

Answer 49 · 2020-06-20T23:40:19.000Z

Changed commit from daa045c to e3d9cf2

Answer 50 · 2020-06-20T23:40:54.000Z

comment:46

(these scripts are deleted in #29920)

Answer 51 · 2020-06-20T23:50:35.000Z

comment:47

A bit much to ask but is there a way to override the scripts variable in setup.py? Apart from patching that is. I and other distros will pick and choose what we install and certainly, even where. We can do things after install is run if needs be.

But building and providing our own list would be a nice to have.

Answer 52 · 2020-06-20T23:56:01.000Z

comment:48

Replying to @kiwifb:

A bit much to ask but is there a way to override the scripts variable in setup.py? Apart from patching that is. I and other distros will pick and choose what we install and certainly, even where. We can do things after install is run if needs be.

But building and providing our own list would be a nice to have.

Adding hooks for distributors is not a good direction for development. It's better to sort out in sagelib itself which scripts should be considered part of sagelib, and which ones are provided by sage-the-distribution instead.

Answer 53 · 2020-06-20T23:58:39.000Z

comment:49

This can be done on follow-up tickets of this ticket.
You can help by categorizing the scripts in the section "Uncategorized scripts in alphabetical order".

Answer 54 · 2020-06-21T00:01:33.000Z

comment:50

Replying to @mkoeppe:

This can be done on follow-up tickets of this ticket.
You can help by categorizing the scripts in the section "Uncategorized scripts in alphabetical order".

I'll think about it. Like I said it is a "nice to have", not a must have. But this will mark a transition for distros. Before this, most of us were manually installing a selection of scripts. Suddenly the install won't be manual anymore :)

Answer 55 · 2020-06-21T00:19:03.000Z

comment:51

In fact, the scripts that belong to sage-the-distribution could perhaps be assigned to sage_setup (#29847) -- this distutils package that would be installed by sage-the-distribution but not by distros.

Answer 56 · 2020-06-21T02:17:13.000Z

comment:52

Replying to @mkoeppe:

This can be done on follow-up tickets of this ticket.
You can help by categorizing the scripts in the section "Uncategorized scripts in alphabetical order".

Here are some thoughts:

Perhaps could just be deleted:

sage-cython: #29923
sage-native-execute: look at the script and see #9386.
sage-open: I wonder if the comment in that script is still valid; perhaps it can be deleted, or done purely in Python in sage/misc/viewer.py, the only place it's used. Maybe I'll look into this.
sage-pypkg-location: apparently unused
sage-unzip is in the listed, but the script itself is gone, isn't it?

Doctesting-related scripts. Where do these go?

sage-runtests
coverage scripts
sage-fix-doctests
sage-startuptime

In the libexec group:

sage-cleaner?

Only relevant for sage-the-distribution:

sage-download
sage-clone-source
sage-sdist
sage-location

Answer 57 · 2020-06-21T02:31:53.000Z

comment:53

Replying to @jhpalmieri:

Doctesting-related scripts. Where do these go?

sage-runtests

coverage scripts

sage-fix-doctests

Per #29111, they are sagelib scripts.

sage-startuptime

Not sure about this one.

Answer 58 · 2020-06-21T02:33:03.000Z

Changed commit from e3d9cf2 to 455c294

Answer 59 · 2020-06-21T02:33:03.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`455c294`	`src/setup.py: Do not install removed script bin/sage-unzip`

Answer 60 · 2020-06-21T02:40:55.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`b214410`	`Classify doctesting-related scripts`

Answer 61 · 2020-06-21T02:40:55.000Z

Changed commit from 455c294 to b214410

Answer 62 · 2020-06-21T02:45:06.000Z

Changed commit from b214410 to 8d10bb9

Answer 63 · 2020-06-21T02:45:06.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`8d10bb9`	`Categorize some uncategorized scripts`

Answer 64 · 2020-06-23T21:48:28.000Z

Changed commit from 8d10bb9 to 6b04075

Answer 65 · 2020-06-23T21:48:28.000Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`05ba86a`	`Merge branch 't/29884/src_doc_bootstrap__generate_src_doc_en_reference_repl_options_rst' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution`
`72c7e23`	`src/doc/en/reference/repl/options.rst: Replace copypasta by include of generated file options.txt`
`e9a883d`	`trac 29111: revising "sage --advanced" message.`
`5dca421`	`trac 29111: more reorganization of "sage --advanced" message`
`8f0ee05`	`trac 29111: delete sage-fix-pkg-checksums`
`21fc231`	`trac 29111: re "sage --advanced" message:`
`875940d`	`trac 29111: more tinkering. Fix doctests in tests/cmdline.py.`
`4a3d36e`	`Move 'sage -app' back to src/bin/sage`
`3a0193c`	`src/bin/sage: Remove handling of 'sage -axiom'`
`6b04075`	`Merge branch 't/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution' into t/21559/change-src-bin-installation`

Answer 66 · 2020-06-23T21:51:04.000Z

Changed commit from 6b04075 to 831cc09

Answer 67 · 2020-06-23T21:51:04.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`9c7116b`	`src/bin/sage-list-optional, sage-list-experimental, sage-list-standard: Remove deprecated scripts`
`831cc09`	`Merge branch 't/29920/remove_deprecated_scripts_sage_list_optional__sage_list_experimental__sage_list_standard' into t/21559/change-src-bin-installation`

Answer 68 · 2020-06-23T21:51:29.000Z

Changed commit from 831cc09 to e3eca85

Answer 69 · 2020-06-23T21:51:29.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`a56dc35`	`Merge tag '9.2.beta1' into t/29702/public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup`
`e3eca85`	`Merge branch 'public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' of git://trac.sagemath.org/sage into t/21559/change-src-bin-installation`

Answer 70 · 2020-06-24T16:24:11.000Z

comment:60

I run into problems unless I make this change:

diff --git a/src/setup.py b/src/setup.py
index 9970525cf9..85a32a5962 100755
--- a/src/setup.py
+++ b/src/setup.py
@@ -162,7 +162,6 @@ code = setup(name = 'sage',
                  'bin/sage-rebaseall.sh',
                  'bin/sage-rst2sws',
                  'bin/sage-rst2txt',
-                 'bin/sage-rsyncdist',
                  'bin/sage-run',
                  'bin/sage-run-cython',
                  'bin/sage-startuptime.py',

Answer 71 · 2020-06-24T18:26:33.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`7d29141`	`src/setup.py: Do not install removed script sage-rsyncdist`

Answer 72 · 2020-06-24T18:26:33.000Z

Changed commit from e3eca85 to 7d29141

Answer 73 · 2020-06-25T04:34:31.000Z

Description changed:

--- 
+++ 
@@ -1,5 +1,9 @@
 This ticket cleans up how the contents of `src/bin` are installed. One of its goals is to make the `sagelib` installation done by `src/setup.py` self-contained regarding the scripts that it needs at runtime.
 
-Previous tickets moved some of the scripts to `build/bin` (no longer installed in `SAGE_LOCAL`). In follow-up tickets, `src/bin` could be cleaned up more.  See the list of files added on this ticket to `src/setup.py` for beginnings of a categorization.
- 
+Previous tickets moved some of the scripts to `build/bin` (no longer installed in `SAGE_LOCAL`). 
 
+In follow-up tickets, `src/bin` could be cleaned up more. 
+- See the list of files added on this ticket to `src/setup.py` for beginnings of a categorization.
+- #29850: Install `src/bin/sage-env-config` as part of `sage_conf`
+- #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"
+

Answer 74 · 2020-06-29T17:33:58.000Z

comment:63

Needs review

Answer 75 · 2020-06-29T21:30:44.000Z

comment:64

Looks okay to me.

Answer 76 · 2020-06-29T21:30:44.000Z

Reviewer: John Palmieri, François Bissey

Answer 77 · 2020-06-30T00:14:45.000Z

comment:65

Thanks!

Answer 78 · 2020-07-04T16:20:24.000Z

Changed commit from 7d29141 to c5f7f2f

Answer 79 · 2020-07-04T16:20:24.000Z

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

`c5f7f2f`	`Merge tag '9.2.beta3' into t/21559/change-src-bin-installation`

Answer 80 · 2020-07-08T23:22:53.000Z

Changed commit from c5f7f2f to 19d7f67

Answer 81 · 2020-07-08T23:22:53.000Z

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

`3953671`	`Merge tag '9.2.beta4' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution`
`19d7f67`	`Merge branch 't/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution' into t/21559/change-src-bin-installation`

Answer 82 · 2020-07-08T23:23:26.000Z

Changed dependencies from #29702, #29111, #29920 to #29111

Answer 83 · 2020-07-10T19:34:13.000Z

Changed branch from u/mkoeppe/change-src-bin-installation to 19d7f67