./configure --prefix=SAGE_LOCAL
Closed this issue · 157 comments
I propose to support choosing a location for the SAGE_LOCAL tree, using
./configure --prefix=SAGE_LOCAL
(the default would be, as before, the local subdirectory of SAGE_ROOT - see patch on the ticket).
I am fully aware that Sage's build system does not have a separation between 'make' and 'make install'; see #21495 for that.
Nevertheless, SAGE_LOCAL should be considered the same as the prefix in the sense of the autotools.
This ticket is a step towards #21566.
Release manager configure tarball: http://sage.ugent.be/www/jdemeyer/sage/configure-214.tar.gz
CC: @jdemeyer @embray @kiwifb @nexttime @dimpase
Component: build
Author: Matthias Koeppe, Jeroen Demeyer
Branch: e5926b1
Reviewer: Jeroen Demeyer, Erik Bray, Matthias Koeppe
Issue created by migration from https://trac.sagemath.org/ticket/21479
Description changed:
---
+++
@@ -3,7 +3,7 @@
```
./configure --prefix=SAGE_LOCAL
```
-(the default would be, as before, the `local` subdirectory of SAGE_ROOT).
+(the default would be, as before, the `local` subdirectory of SAGE_ROOT - see patch on the ticket).
I am fully aware that Sage's build system does not have a separation between 'make' and 'make install' (this would require all packages to support a two-stage install (DESTDIR).) Nevertheless, SAGE_LOCAL should be considered the same as the `prefix` in the sense of the autotools.
Author: Matthias Koeppe
Description changed:
---
+++
@@ -5,6 +5,8 @@
```
(the default would be, as before, the `local` subdirectory of SAGE_ROOT - see patch on the ticket).
-I am fully aware that Sage's build system does not have a separation between 'make' and 'make install' (this would require all packages to support a two-stage install (DESTDIR).) Nevertheless, SAGE_LOCAL should be considered the same as the `prefix` in the sense of the autotools.
+I am fully aware that Sage's build system does not have a separation between 'make' and 'make install'; see #21480 for that.
+
+Nevertheless, SAGE_LOCAL should be considered the same as the `prefix` in the sense of the autotools.
Description changed:
---
+++
@@ -5,7 +5,7 @@
```
(the default would be, as before, the `local` subdirectory of SAGE_ROOT - see patch on the ticket).
-I am fully aware that Sage's build system does not have a separation between 'make' and 'make install'; see #21480 for that.
+I am fully aware that Sage's build system does not have a separation between 'make' and 'make install'; see #21495 for that.
Nevertheless, SAGE_LOCAL should be considered the same as the `prefix` in the sense of the autotools.
(i completely agree with this)
a main obstacle is: the build system writes to $prefix during the build...
i suggest to do the transition one-package-at-a-time, and after --disable-$package is in place.
with --disable-all, it will be easiest to avoid writing to $prefix.
On this short term ticket, we do allow "make" to write into $prefix; and "make install" will be a no-op.
More ambitious plans are to be discussed on the long term ticket #21495, not on this ticket. Thanks.
i agree that there are intermediate steps to take. but i don't yet fully understand this approach.
i think the behaviour of just "make" should not change, regardless of --prefix. it works perfectly well and as intended right now. clearly it should do something different under the hood, but that's not on this ticket.
we do allow "make" to write into $prefix
why would you need/want that? can you please give an example?
my feeling is, that entangling prefix and SAGE_LOCAL further complicates the transition considerably. there will be no way to tell whether "this part still uses SAGE_LOCAL" vs. "this has already been cleaned". newcomers tend to grep a variable from the code, see how it is used and use it the same way. for a looong time to come.
Changed author from Matthias Koeppe to Matthias Koeppe, Jeroen Demeyer
Description changed:
---
+++
@@ -8,5 +8,3 @@
I am fully aware that Sage's build system does not have a separation between 'make' and 'make install'; see #21495 for that.
Nevertheless, SAGE_LOCAL should be considered the same as the `prefix` in the sense of the autotools.
-
-Changed author from Matthias Koeppe, Jeroen Demeyer to Matthias Koeppe
i wrote
entangling prefix and SAGE_LOCAL further complicates the transition considerably.
@mkoeppe i've thought about it. it seems, the alternatives are much worse (#21501) or much more ambitious.
please go ahead with prefix==SAGE_LOCAL.
please consider to add a remark in some documentation (better place: configure --help?) on this temporary sort of prefix, such as
"currently, make install is a no-op, that will change (hopefully). you should not expect a fully functional installation in $prefix before make install has finished."
Branch pushed to git repo; I updated commit sha1. New commits:
64e697d | Allow a custom $SAGE_LOCAL directory |
dc81cf6 | Merge remote-tracking branch 'trac/u/jdemeyer/ticket/21501' into t/21479/__configure___prefix_sage_local |
ac12edc | Merge tag '7.4.beta5' into t/21479/__configure___prefix_sage_local |
924b30b | Handle --prefix |
Branch pushed to git repo; I updated commit sha1. New commits:
6f8eaac | Fixup |
Here's a first version of what I have in mind.
Some concerns:
- bootstrap. It's somewhat unclear what to do here. By the way, is it supposed to use the distribution's Python or ours?
- sage-location.
I don't like the duplication in build/make/install and src/bin/sage-env.
Here is a suggestion: add a new file, say src/bin/sage-config (or whatever you want to call it) to deal with configurable things like SAGE_LOCAL. You can also define the other SAGE_* directories (except SAGE_ROOT) there.
Then you can source this file in both src/bin/sage-env and build/make/install, which can then be independent of configure.
And I don't see the point of changing $SAGE_ROOT/sage because that is additional duplication.
Replying to @mkoeppe:
Some concerns:
- bootstrap. It's somewhat unclear what to do here. By the way, is it supposed to use the distribution's Python or ours?
- sage-location.
In both cases, I don't get what the problem is.
Replying to @jdemeyer:
Here is a suggestion: add a new file, say
src/bin/sage-config(or whatever you want to call it) to deal with configurable things likeSAGE_LOCAL. You can also define the otherSAGE_*directories (exceptSAGE_ROOT) there.
Thanks for the suggestion.
The ticket is indeed simpler that way.
Please take a look.
Branch pushed to git repo; I updated commit sha1. New commits:
e0dc9a3 | Merge tag '7.4.beta6' into t/21479/__configure___prefix_sage_local |
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
751bd0f | Reword TODO item |
3a8cc0e | Fix typo in comment |
0dd9c50 | Respect environment variable MAKE |
17f90d8 | beautification |
e5f9065 | More comments |
7791cd9 | Remove --buildbase code |
74169e7 | Pass SAGE_SRC to generate_py_source.mk |
0394333 | Add new file to MANIFEST.in |
fdedb02 | Install SAGE_LOCAL/bin/sage instead of SAGE_ROOT/sage as Jupyter kernel |
8215254 | Merge branch 't/21480/keep_src__clean_by_using___build_base_when_building_sagelib' into t/21479/__configure___prefix_sage_local |
Description changed:
---
+++
@@ -8,3 +8,5 @@
I am fully aware that Sage's build system does not have a separation between 'make' and 'make install'; see #21495 for that.
Nevertheless, SAGE_LOCAL should be considered the same as the `prefix` in the sense of the autotools.
+
+This ticket is a step towards #21566.Replying to @mkoeppe:
Here's a first version of what I have in mind.
Some concerns:
- bootstrap. It's somewhat unclear what to do here. By the way, is it supposed to use the distribution's Python or ours?
Just to answer this question: the bootstrap package is intended to use the system Python (after all, it's needed well before Sage's Python is ever built).
Branch pushed to git repo; I updated commit sha1. New commits:
06fe631 | Merge tag '7.4.rc1' into t/21479/__configure___prefix_sage_local |
Branch is now on top of 7.4.rc1.
Jeroen - ready for review.
Branch pushed to git repo; I updated commit sha1. New commits:
1adc18c | Merge tag '7.4' into t/21479/__configure___prefix_sage_local |
Now on top of 7.4.
Branch pushed to git repo; I updated commit sha1. New commits:
d82cfbc | Merge tag '7.5.beta0' into t/21479/__configure___prefix_sage_local |
What does this comment mean:
## The first test does not actually work (see above).
("see above" can be really confusing because you don't know where to look.
Comments like these don't belong in the sources but in a Trac ticket:
# TODO: Making directories in $prefix
# should be done by 'make', not configure.
# TODO: Should the install hierarchy (local) really be deleted by make distclean?
#SAGE_ENV_DIR=`dirname $0` ### Does not work. $0 is /bin/bash
## Note this is a bashism, not sure if this is allowed in this file.
Then a more serious remark: I doubt that you are using BASH_SOURCE correctly. From the bash man page:
BASH_SOURCE
An array variable whose members are the source filenames where the corresponding shell function names in the FUNCNAME array variable are
defined. The shell function ${FUNCNAME[$i]} is defined in the file ${BASH_SOURCE[$i]} and called from ${BASH_SOURCE[$i+1]}.
At the very least, I need more explanation of why your code does the right thing.
Replying to @jdemeyer:
What does this comment mean:
## The first test does not actually work (see above).("see above" can be really confusing because you don't know where to look.
It's referring to dirname $0 -- which does not give the location of sage-env but rather /bin/bash. This is because sage-env is sourced (.) rather than executed.
Note this is existing code in sage-env which simply never worked.
Comments like these don't belong in the sources but in a Trac ticket:
I know, these comments are for the reviewer. I'll be happy remove them after we had a discussion about them... So do you have a comment on these?
# TODO: Making directories in $prefix # should be done by 'make', not configure.# TODO: Should the install hierarchy (local) really be deleted by make distclean?#SAGE_ENV_DIR=`dirname $0` ### Does not work. $0 is /bin/bash## Note this is a bashism, not sure if this is allowed in this file.
Replying to @jdemeyer:
Then a more serious remark: I doubt that you are using
BASH_SOURCEcorrectly. From thebashman page:BASH_SOURCE An array variable whose members are the source filenames where the corresponding shell function names in the FUNCNAME array variable are defined. The shell function ${FUNCNAME[$i]} is defined in the file ${BASH_SOURCE[$i]} and called from ${BASH_SOURCE[$i+1]}.At the very least, I need more explanation of why your code does the right thing.
BASH_SOURCE[0] gives the source of the "currently running function". See https://www.gnu.org/software/bash/manual/html_node/Bash-Variables.html (you need to read the entries for FUNCNAME and BASH_SOURCE).
Replying to @mkoeppe:
Replying to @jdemeyer:
What does this comment mean:
## The first test does not actually work (see above).("see above" can be really confusing because you don't know where to look.
It's referring to
dirname $0-- which does not give the location of sage-env but rather /bin/bash. This is because sage-env is sourced (.) rather than executed.
Don't tell me but tell the person reading sage-env (i.e. clarify the comment in the file).
I know, these comments are for the reviewer. I'll be happy remove them after we had a discussion about them... So do you have a comment on these?
Comments for the reviewer can go on Trac.
OK, done; please take a look
Branch pushed to git repo; I updated commit sha1. New commits:
4275121 | Installation manual: Document configure --prefix |
Branch pushed to git repo; I updated commit sha1. New commits:
ea994cc | Merge tag '7.5.beta1' into t/21479/__configure___prefix_sage_local |
ping?
Running ./configure without arguments:
[...]
config.status: creating build/make/Makefile-auto
config.status: creating src/Makefile
config.status: creating src/bin/sage-env-config
config.status: executing depfiles commands
config.status: executing mkdirs commands
config.status: creating directory /usr/local/src/sage-config/logs/pkgs
config.status: creating directory NONE
config.status: creating directory NONE/bin
config.status: creating directory NONE/etc
config.status: creating directory NONE/include
config.status: creating directory NONE/lib
config.status: creating directory NONE/share
config.status: creating directory NONE/var/lib/sage/installed
config.status: creating symbolic link lib64 -> lib
Reviewer: Jeroen Demeyer
This is just because you write SAGE_LOCAL instead of "$SAGE_LOCAL".
When you fix this, I suggest to squash the commits to one commit and rebase to 7.5.beta1.
The file sage-env-config needs to be ignored by .gitignore and deleted when running make distclean.
This line in sage needs quoting:
. $SAGE_ROOT/src/bin/sage-env-config
And since you're running bash anyway, you might as well use source instead of . which is more visible.
And do you really need this:
if [ $? -ne 0 ]; then
echo >&2 "Error: $SAGE_ROOT/src/bin/sage-env-config could not be read"
fi
The shell will already write an error message, no need for an additional message.
I also think that determining $SAGE_LOCAL should be done after
# If this is a freshly-unpacked binary tarball then run the installer
# Note: relocate-once.py deletes itself upon successful completion
if [ -x "$SAGE_ROOT/relocate-once.py" ]; then
"$SAGE_ROOT/relocate-once.py"
fi
Can you explain under which circumstances it is needed to read sage-env-config from sage-env? It seems to me that sage-env-config would already be read, either by the top-level sage script or by build/make/install.
It is good style to start all error messages with Error: and write them to stderr. build/make/install doesn't do that.
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
4f832c2 | Add configure option --prefix=SAGE_LOCAL |
Squashed and rebased on 7.5.beta1
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
6308c3b | Add configure option --prefix=SAGE_LOCAL |
ff48d5a | Fixup configure without --prefix |
67baa7a | Source sage-env-config after calling relocate-once.py |
5fcda83 | top-level sage script: Remove redundant sage-env-config error message |
Branch pushed to git repo; I updated commit sha1. New commits:
600639a | sage-env: Quoting fix |
Replying to @jdemeyer:
Can you explain under which circumstances it is needed to read
sage-env-configfromsage-env? It seems to me thatsage-env-configwould already be read, either by the top-levelsagescript or bybuild/make/install.
$SAGE_LOCAL/bin/sage becomes directly executable; one does not have to go through that top-level script.
Replying to @jdemeyer:
since you're running
bashanyway, you might as well usesourceinstead of.which is more visible.
I'm not sure, I like . more than source.
Replying to @mkoeppe:
$SAGE_LOCAL/bin/sagebecomes directly executable; one does not have to go through that top-level script.
Then let me turn the question around: why do you need to change $SAGE_ROOT/sage?
I'm still not entirely convinced that BASH_SOURCE[0] really does what we want. Why not use $SAGE_ROOT/src/bin/sage-env-config?
Or maybe we should just source both sage-env-config and sage-env from src/bin/sage?
Replying to @jdemeyer:
Why not use
$SAGE_ROOT/src/bin/sage-env-config?
Things installed in SAGE_LOCAL definitely should not depend on the source directory.
Or maybe we should just source both
sage-env-configandsage-envfromsrc/bin/sage?
That would be possible. There might be a few more places though that need to source the extra file.
Replying to @jdemeyer:
Replying to @mkoeppe:
$SAGE_LOCAL/bin/sagebecomes directly executable; one does not have to go through that top-level script.Then let me turn the question around: why do you need to change
$SAGE_ROOT/sage?
$SAGE_ROOT/sage sources $SAGE_ROOT/src/bin/sage-env-config so that it works when nothing has been installed so far.
Replying to @mkoeppe:
$SAGE_ROOT/sagesources$SAGE_ROOT/src/bin/sage-env-configso that it works when nothing has been installed so far.
Sorry, I don't follow. What is the "it" in the sentence above? In other words, what would not work?
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
f6c4513 | Add configure option --prefix=SAGE_LOCAL |
7d44f0b | Fixup configure without --prefix |
387ab94 | Source sage-env-config after calling relocate-once.py |
fc66139 | top-level sage script: Remove redundant sage-env-config error message |
fe19d3f | src/bin/sage-env-config: Prefix error message with 'Error:' |
269ad80 | make misc-clean: Delete src/bin/sage-env-config |
727123a | Add sage-env-config to .gitignore |
be4c390 | sage-env: Quoting fix |
a3467bf | top-level sage script: No need to source sage-env-config |
Replying to @jdemeyer:
Replying to @mkoeppe:
$SAGE_ROOT/sagesources$SAGE_ROOT/src/bin/sage-env-configso that it works when nothing has been installed so far.Sorry, I don't follow. What is the "it" in the sentence above? In other words, what would not work?
I retract my comment.
I have removed this code from $SAGE_ROOT/sage.
(Also rebased on latest beta.)
Replying to @mkoeppe:
Or maybe we should just source both
sage-env-configandsage-envfromsrc/bin/sage?That would be possible. There might be a few more places though that need to source the extra file.
I think I wouldn't like to change this on this ticket. There are indeed a few places that source sage-env: build/bin/sage-spkg, build/make/deps, src/bin/sage-update-src. Probably not so nice if we have to change all of them.
Replying to @mkoeppe:
$SAGE_LOCAL/bin/sagebecomes directly executable; one does not have to go through that top-level script.
I realized that this is not entirely true. sage-env contains some code for discovering SAGE_ROOT when it is not set, but that code fails if $SAGE_LOCAL is not a direct subdirectory of $SAGE_ROOT.
I suppose I could add the definition of SAGE_ROOT to sage-env-config.
But that could go on a follow-up ticket as well.
ping
Changed reviewer from Jeroen Demeyer to Jeroen Demeyer, Erik Bray
Doesn't work if you dont have autotools
19make -j2 build/make/Makefile
20make[1]: Entering directory `/home/buildbot/slave/sage_git/build'
21make[1]: warning: -jN forced in submake: disabling jobserver mode.
22./bootstrap -d
23make[2]: Entering directory `/home/buildbot/slave/sage_git/build'
24rm -rf config configure build/make/Makefile-auto.in
25make[2]: Leaving directory `/home/buildbot/slave/sage_git/build'
26src/bin/sage-env: line 206: /home/buildbot/slave/sage_git/build/src/bin/sage-env-config: No such file or directory
27sage-env: Error: /home/buildbot/slave/sage_git/build/src/bin/sage-env-config could not be read.
28./bootstrap: line 31: aclocal: command not found
29Bootstrap failed, downloading required files instead.
30./bootstrap: line 61: sage-download-file: command not found
31Error: downloading configure-200.tar.gz failed
32make[1]: *** [configure] Error 1
33make[1]: Target `build/make/Makefile' not remade because of errors.
34make[1]: Leaving directory `/home/buildbot/slave/sage_git/build'
35make: *** [start] Error 2
36program finished with exit code 2
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
928f67d | Add configure option --prefix=SAGE_LOCAL |
66a8066 | Fixup configure without --prefix |
b5ecc81 | Source sage-env-config after calling relocate-once.py |
c1c36ff | top-level sage script: Remove redundant sage-env-config error message |
ae6263b | src/bin/sage-env-config: Prefix error message with 'Error:' |
27189be | make misc-clean: Delete src/bin/sage-env-config |
eb922b8 | Add sage-env-config to .gitignore |
4cdaa43 | sage-env: Quoting fix |
71f88cc | top-level sage script: No need to source sage-env-config |
52826b1 | bootstrap: Don't use sage-env, set PATH directly |
Replying to @vbraun:
Doesn't work if you dont have autotools
Thanks for catching this.
Fixed. Needs review.
Erik, if you could take another look...
I'm not really clear about what changed. I don't understand what the issue with autotools was or how this was fixed. I see that you did do something to address that but I don't understand the original issue in the first place--would be better if Volker commented since he pointed it out.
Jeroen, could you take another look?
This ticket needs review.
pong
Why does this require changes to bootstrap anyway?