sagemath/sage

sage.env: Remove direct uses of SAGE_LOCAL from the Sage library

Closed this issue · 110 comments

(from #31338 comment:13)

These changes are preparation for modularization and is also intended to make downstream packaging easier.

CC: @kiwifb @antonio-rojas @dimpase @videlec @jhpalmieri

Component: build: configure

Author: Matthias Koeppe, François Bissey

Branch/Commit: 985af6e

Reviewer: Dima Pasechnik, François Bissey

Issue created by migration from https://trac.sagemath.org/ticket/32036

comment:1

There are a few remaining uses of SAGE_SHARE in src/sage that will have to be eliminated though.

comment:2

(see #30819 sage.databases: Use sage.feature to define database file system location)

comment:3

Perhaps now is also the time that we do #30914 (Meta-ticket: Create upstream repositories, pip-installable packages for database packages)?

comment:4

Replying to @mkoeppe:

Perhaps now is also the time that we do #30914 (Meta-ticket: Create upstream repositories, pip-installable packages for database packages)?

That would be timely if we could.

Other issue about sage_setup/setenv.py I tried to replace if SAGE_LOCAL: by if SAGE_ROOT: since it is set to None for me in distro. But I still got the linking noise which means that it still accepted that as a positive. So a better test may be needed.

comment:5

No, SAGE_ROOT is not the right choice there.

comment:6

Replying to @kiwifb:

Other issue about sage_setup/setenv.py I tried to replace if SAGE_LOCAL: by if SAGE_ROOT: since it is set to None for me in distro. But I still got the linking noise which means that it still accepted that as a positive. So a better test may be needed.

Scratch that, I think I did myself in at build time for some reasons (which I have top check).

Commit: 95deabd

New commits:

95deabdsrc/sage/env.py: No fallback for SAGE_LOCAL

Author: Matthias Koeppe

comment:9
> sage
Traceback (most recent call last):
  File "/usr/bin/sage-ipython", line 9, in <module>
    from sage.misc.banner import banner
  File "/usr/lib/python3.9/site-packages/sage/misc/banner.py", line 16, in <module>
    from sage.env import (SAGE_VERSION, SAGE_VERSION_BANNER, SAGE_BANNER)
  File "/usr/lib/python3.9/site-packages/sage/env.py", line 314, in <module>
    SINGULAR_SO = var("SINGULAR_SO", _get_shared_lib_path("Singular", "singular-Singular"))
  File "/usr/lib/python3.9/site-packages/sage/env.py", line 291, in _get_shared_lib_path
    search_directories = [Path(SAGE_LOCAL) / 'lib']
  File "/usr/lib/python3.9/pathlib.py", line 1072, in __new__
    self = cls._from_parts(args, init=False)
  File "/usr/lib/python3.9/pathlib.py", line 697, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/usr/lib/python3.9/pathlib.py", line 681, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Changed commit from 95deabd to de66da4

Branch pushed to git repo; I updated commit sha1. New commits:

de66da4sage.env._get_shared_lib_path: If SAGE_LOCAL is not set, skip it
comment:11

Thanks for catching this!

comment:12

Next issue:

/usr/lib/python3.9/site-packages/sage/interfaces/gap_workspace.py in gap_workspace_file(system='gap', name='workspace', dir='/home/antonio/.sage/gap')
     [...]
     57         sage: assert name1 == name2
     58     """
     59     if dir is None:
     60         dir = os.path.join(DOT_SAGE, 'gap')
     61 
---> 62     h = hashlib.sha1(SAGE_LOCAL.encode('utf-8')).hexdigest()
        h = undefined
        global hashlib.sha1 = <built-in function openssl_sha1>
        global SAGE_LOCAL.encode.hexdigest = undefined
     63     return os.path.join(dir, '%s-%s-%s' % (system, name, h))
     64 
     65 
AttributeError: 'NoneType' object has no attribute 'encode'

and, after bypassing it

/usr/lib/python3.9/site-packages/sage/interfaces/lie.py in __init__(self=LiE Interpreter, maxread=None, script_subdirectory=None, logfile=None, server=None)
    [...]
    329                         # it to be very obfuscated that would be better.   Even
    330                         # better is to use sequence numbers.
    331                         prompt = '> ',
    332 
    333                         # This is the command that starts up your program
--> 334                         command = "bash "+ SAGE_LOCAL + "/bin/lie",
        global command = undefined
        global SAGE_LOCAL = None
    335 
    336                         server=server,
    337                         script_subdirectory = script_subdirectory,
    338 

TypeError: can only concatenate str (not "NoneType") to str
comment:13

Excluding env.py for a minute those are the instances of SAGE_LOCAL still existing under in the sagelib tree from a simple grep command

src/sage/all.py:from sage.env import SAGE_ROOT, SAGE_SRC, SAGE_DOC_SRC, SAGE_LOCAL, DOT_SAGE, SAGE_ENV
src/sage/all.py:        sage: started_file = os.path.join(SAGE_LOCAL, 'etc', 'sage-started.txt')
src/sage/all.py:    started_file = os.path.join(SAGE_LOCAL, 'etc', 'sage-started.txt')
src/sage/calculus/desolvers.py:                          os.path.join('$SAGE_LOCAL','lib','libTIDES.a') + ' $LDFLAGS '
src/sage/calculus/desolvers.py:                          + os.path.join('-L$SAGE_LOCAL','lib ') +' -lm  -O2 ' +
src/sage/calculus/desolvers.py:                          os.path.join('-I$SAGE_LOCAL','include '),
src/sage/calculus/desolvers.py:                          os.path.join('$SAGE_LOCAL','lib','libTIDES.a') + ' $LDFLAGS '
src/sage/calculus/desolvers.py:                          + os.path.join('-L$SAGE_LOCAL','lib ') + '-lmpfr -lgmp -lm  -O2 -w ' +
src/sage/calculus/desolvers.py:                          os.path.join('-I$SAGE_LOCAL','include ') ,
src/sage/doctest/control.py:        # only have the SAGE_LOCAL install tree but not SAGE_ROOT
src/sage/doctest/control.py:            # only have the SAGE_LOCAL install tree but not SAGE_ROOT
src/sage/interfaces/qepcad.py:    sage: with open(os.path.join(SAGE_LOCAL, 'default.qepcadrc')) as f:  # optional - qepcad
src/sage/interfaces/qepcad.py:from sage.env import SAGE_LOCAL
src/sage/interfaces/qepcad.py:        sage: s == 'env qe=%s qepcad '%SAGE_LOCAL
src/sage/interfaces/qepcad.py:        sage: s == 'env qe=%s qepcad +N8000000'%SAGE_LOCAL
src/sage/interfaces/qepcad.py:    return "env qe=%s qepcad %s"%(SAGE_LOCAL, memcells_arg)
src/sage/interfaces/qepcad.py:    with open(os.path.join(SAGE_LOCAL, 'bin', 'qepcad.help')) as help:
src/sage/interfaces/giac.py:    $SAGE_LOCAL/share/giac/doc/en/cascmd_local/index.html
src/sage/interfaces/giac.py:If you got giac from the spkg then ``$PREFIX`` is ``$SAGE_LOCAL``
src/sage/interfaces/gap_workspace.py:from sage.env import DOT_SAGE, SAGE_LOCAL
src/sage/interfaces/gap_workspace.py:    h = hashlib.sha1(SAGE_LOCAL.encode('utf-8')).hexdigest()
src/sage/interfaces/lie.py:from sage.env import DOT_SAGE, SAGE_LOCAL
src/sage/interfaces/lie.py:                        command = "bash "+ SAGE_LOCAL + "/bin/lie",
src/sage/interfaces/lie.py:            filename = SAGE_LOCAL + "/lib/LiE/" + f
src/sage/interfaces/lie.py:    f = open(os.path.join(SAGE_LOCAL, 'lib', 'LiE', 'INFO.0'))
src/sage/interfaces/maxima.py:from sage.env import DOT_SAGE, SAGE_LOCAL, MAXIMA
src/sage/interfaces/mathematica.py:##            it in <SAGE_LOCAL>/bin/ is an easy way to ensure this).
src/sage/libs/gap/util.pyx:    SAGE_LOCAL = sage.env.SAGE_LOCAL
src/sage/libs/gap/util.pyx:    with open(os.path.join(SAGE_LOCAL, 'bin', 'gap')) as f:
src/sage/libs/gap/util.pyx:    gapdir = gapdir.replace('$SAGE_LOCAL', SAGE_LOCAL)
src/sage/libs/giac/giac.pyx:      - or in :doc:`$SAGE_LOCAL/share/giac/doc/en/cascmd_en/index.html`
src/sage/misc/sage_ostools.pyx:        sage: have_program('there_is_not_a_program_with_this_name', os.path.join(SAGE_LOCAL, 'bin'))
src/sage/misc/citation.pyx:from sage.env import SAGE_LOCAL
src/sage/misc/citation.pyx:    strings = [a[0].replace(SAGE_LOCAL, "") + " " + a[2]
src/sage/misc/cython.py:from sage.env import (SAGE_LOCAL, cython_aliases,
src/sage/misc/cython.py:    standard_libdirs = [os.path.join(SAGE_LOCAL, "lib")] + aliases["CBLAS_LIBDIR"] + aliases["NTL_LIBDIR"]
src/sage/misc/compat.py:from sage.env import SAGE_LOCAL
src/sage/misc/compat.py:        for libdir in [os.path.join(SAGE_LOCAL, 'lib'),
src/sage/misc/compat.py:        sage_local_lib = os.path.join(SAGE_LOCAL, 'lib')
src/sage/misc/compat.py:    lib_dirs = (LDPATH_STR.split(':') if LDPATH_STR else []) + [os.path.join(SAGE_LOCAL, 'lib')]
src/sage/misc/edit_module.py:    'SAGE_LOCAL/lib/python.../site-packages', it replaces this by
src/sage/misc/dist.py:    from sage.env import SAGE_LOCAL
src/sage/misc/dist.py:    SAGE_BIN = os.path.join(SAGE_LOCAL, 'bin')
src/sage/misc/dist.py:    # SAGE_LOCAL/bin from PATH so that we can later check whether
src/sage/plot/plot3d/base.pyx:from sage.env import SAGE_LOCAL
src/sage/repl/interpreter.py:``$SAGE_LOCAL/bin/sage-ipython`` script to start the Sage
src/sage/sandpiles/sandpile.py:from sage.env import SAGE_LOCAL
src/sage/sandpiles/sandpile.py:path_to_zsolve = os.path.join(SAGE_LOCAL, 'bin', 'zsolve')
  • sage-started could be eliminated without regrets (personal opinion).
  • The stuff in calculus/desolvers.py should be carefully reviewed - can we still use that stuff and if yes, how?
  • qepcad.py is long time serial offender.
  • gap_workspace.py needs to be looked at.
  • lie.py probably should have specific variable, distro may put the stuff somewhere else (Gentoo does).
  • SAGE_LOCAL is imported in maxima.py and plot/plot3d/base.pyx but appear to be unused - expunge.
  • libs/gap/util.pyx would need to be reworked.
  • cython.py need to be looked at.
  • What is happening with SAGE_LOCAL in misc/compat.py.
  • citations.pyx seem to rely on it.
  • sandpiles/sandpile.py features should be used.

Changed commit from de66da4 to d4b822b

Branch pushed to git repo; I updated commit sha1. New commits:

d4b822bsrc/sage/interfaces/maxima.py: Remove unused imports of DOT_SAGE, SAGE_LOCAL

Changed commit from d4b822b to 13b780b

Branch pushed to git repo; I updated commit sha1. New commits:

13b780bsrc/sage/interfaces/gap_workspace.py: Use hash of GAP_SO to disambiguate the workspace file, not SAGE_LOCAL
comment:16

Perhaps someone who knows more about GAP workspaces can comment whether this change makes sense.

Branch pushed to git repo; I updated commit sha1. New commits:

7ca61a9build/pkgs/lie/distros/*.txt: New

Changed commit from 13b780b to 7ca61a9

comment:18

Replying to @kiwifb:

  • lie.py probably should have specific variable, distro may put the stuff somewhere else (Gentoo does).

I agree.

Could someone on such a distro take care of lie.py please?

There's no package for homebrew and our SPKG does not compile

Changed commit from 7ca61a9 to f1a9c25

Branch pushed to git repo; I updated commit sha1. New commits:

f1a9c25src/sage/misc/sage_ostools.pyx: Remove useless use of SAGE_LOCAL in doctest
comment:20

Replying to @mkoeppe:

Replying to @kiwifb:

  • lie.py probably should have specific variable, distro may put the stuff somewhere else (Gentoo does).

I agree.

Could someone on such a distro take care of lie.py please?

I will.

Branch pushed to git repo; I updated commit sha1. New commits:

faa9637src/sage/misc/dist.py: Return with error message if SAGE_LOCAL is not set

Changed commit from f1a9c25 to faa9637

Branch pushed to git repo; I updated commit sha1. New commits:

82f5c3asrc/sage/misc/compat.py: Do not fail if SAGE_LOCAL is unset

Changed commit from faa9637 to 82f5c3a

comment:23

I am a bit confused by what dist.py does. If it is doing an installation task, should it live in sage_setup?

Changed commit from 82f5c3a to fc47386

Branch pushed to git repo; I updated commit sha1. New commits:

fc47386src/sage/misc/cython.py: Do not fail if SAGE_LOCAL is not set
comment:25

Replying to @kiwifb:

I am a bit confused by what dist.py does. If it is doing an installation task, should it live in sage_setup?

It's an optional installation step, which hopefully can be removed at some point (#30207)

comment:26

Replying to @kiwifb:

  • sage-started could be eliminated without regrets (personal opinion).

I'll remove it (a previous discussion in #26116 also indicated that there is no need for it any more).

Changed commit from fc47386 to d3828ff

Branch pushed to git repo; I updated commit sha1. New commits:

d3828ffsrc/sage/misc/dist.py: Fix typo

Branch pushed to git repo; I updated commit sha1. New commits:

bfa7ac0build/bin/sage-starts, build/make/Makefile.in, src/sage/all.py: Remove use of sage-started.txt file

Changed commit from d3828ff to bfa7ac0

comment:29

Replying to @kiwifb:

  • The stuff in calculus/desolvers.py should be carefully reviewed - can we still use that stuff and if yes, how?

No need to do this on this ticket -- SAGE_LOCAL is only used as an environment variable in a shell command line there, not as a Python variable

Changed commit from bfa7ac0 to 7d6eced

Branch pushed to git repo; I updated commit sha1. New commits:

7d6ecedsrc/sage/plot/plot3d/base.pyx: Remove unused imports, including SAGE_LOCAL

Branch pushed to git repo; I updated commit sha1. New commits:

c6878a2src/sage/misc/citation.pyx: Do not fail if SAGE_LOCAL is unset; handle SAGE_VENV like SAGE_LOCAL

Changed commit from 7d6eced to c6878a2

comment:33

For the remaining items we have separate open tickets listed in #30818

comment:34

Should I still provide a commit for lie here or add a new ticket in #30818?

comment:35

Here is fine, I think

Changed author from Matthias Koeppe to Matthias Koeppe, François Bissey

comment:36

And here is the LiE stuff.


New commits:

44fd912Introduce LIE_INFO_DIR to point to the location of the info files of a LiE install and use it.

Changed commit from c6878a2 to 44fd912

comment:37
+LIE_INFO_DIR = var("LIE_INFO_DIR", join(SAGE_LOCAL, "lib". "LiE"))

typo - there should be a comma

Changed commit from 44fd912 to 405c248

Branch pushed to git repo; I updated commit sha1. New commits:

405c248correct typo (dot to comma)
comment:39

Replying to @mkoeppe:

+LIE_INFO_DIR = var("LIE_INFO_DIR", join(SAGE_LOCAL, "lib". "LiE"))

typo - there should be a comma

Thanks. I will add that all this work will actually further reduce some patching currently occurring in sage-on-gentoo. Which is a good thing^(TM).

comment:40

Still crashing at startup with the current branch, this should probably depend on #30819

/usr/lib/python3.9/site-packages/sage/databases/conway.py in <module>
     21 
     22 from sage.env import CONWAY_POLYNOMIALS_DATA_DIR
     23 
---> 24 _CONWAYDATA = os.path.join(CONWAY_POLYNOMIALS_DATA_DIR, 'conway_polynomials.p')
        global _CONWAYDATA = undefined
        global os.path.join = <function join at 0x7fa811ace8b0>
        global CONWAY_POLYNOMIALS_DATA_DIR = None
     25 _conwaydict = None
     26 
     27 class DictInMapping(Mapping):

/usr/lib/python3.9/posixpath.py in join(a=None, *p=('conway_polynomials.p',))
     73     If any component is an absolute path, all previous path components
     74     will be discarded.  An empty last part will result in a path that
     75     ends with a separator."""
---> 76     a = os.fspath(a)
        a = None
        global os.fspath = <built-in function fspath>
     77     sep = _get_sep(a)
     78     path = a
     79     try:

TypeError: expected str, bytes or os.PathLike object, not NoneType
comment:41

Replying to @kiwifb:

And here is the LiE stuff.

LIE_INFO_DIR = var("LIE_INFO_DIR", join(SAGE_LOCAL, "lib", "LiE"))

This will still break if neither SAGE_LOCAL nor LIE_INFO_DIR are set. Shouldn't it use features too? (or, at the very least, provide a fallback based on sysconfig.get_config_var('LIBDIR'))

comment:42

CONWAY_POLYNOMIALS_DATA_DIR can just be set to the correct location in sage_conf

comment:43

Replying to @antonio-rojas:

Replying to @kiwifb:

And here is the LiE stuff.

LIE_INFO_DIR = var("LIE_INFO_DIR", join(SAGE_LOCAL, "lib", "LiE"))

This will still break if neither SAGE_LOCAL nor LIE_INFO_DIR are set. Shouldn't it use features too? (or, at the very least, provide a fallback based on sysconfig.get_config_var('LIBDIR'))

LiE is an optional package. So, it won't break unless you try to use it. Upstream is actually not installing stuff properly, we shouldn't be looking up info files under lib of any kind.

Using feature is certainly possible and it should start with detecting the executable. The rest is to consult the doc from inside sage and it really should be at the distribution's discretion because it is arbitrary and not something the program itself knows anything about.

comment:44

I have opened #32055 for the task of adding an spkg-configure.m4 for lie. I don't think a fallback is needed

comment:45

I'll add a feature DatabaseConwayPolynomials

comment:46

I don't get sci-mathematics/lie on my Gentoo box, is it not yet in Gentoo?

comment:48

Replying to @mkoeppe:

CONWAY_POLYNOMIALS_DATA_DIR can just be set to the correct location in sage_conf

I think I'm missing something here, when was sage_conf made mandatory even for distro installs? The description says "This default is no longer necessary because sage_conf is now a requirement" but to me it looks the other way around: sage_conf is required because this ticket purposely breaks setups without it.

comment:49

Replying to @mkoeppe:

Sorry, I think my comment reverted your branch change

Last 10 new commits:

faa9637src/sage/misc/dist.py: Return with error message if SAGE_LOCAL is not set
82f5c3asrc/sage/misc/compat.py: Do not fail if SAGE_LOCAL is unset
fc47386src/sage/misc/cython.py: Do not fail if SAGE_LOCAL is not set
d3828ffsrc/sage/misc/dist.py: Fix typo
bfa7ac0build/bin/sage-starts, build/make/Makefile.in, src/sage/all.py: Remove use of sage-started.txt file
7d6ecedsrc/sage/plot/plot3d/base.pyx: Remove unused imports, including SAGE_LOCAL
c6878a2src/sage/misc/citation.pyx: Do not fail if SAGE_LOCAL is unset; handle SAGE_VENV like SAGE_LOCAL
44fd912Introduce LIE_INFO_DIR to point to the location of the info files of a LiE install and use it.
405c248correct typo (dot to comma)
e1eac9fsage.features.DatabaseConwayPolynomials: New, use it in sage.databases.conway

Changed commit from 405c248 to e1eac9f

comment:51

Replying to @dimpase:

I don't get sci-mathematics/lie on my Gentoo box, is it not yet in Gentoo?

No idea, I got this information from https://repology.org/project/lie/versions

comment:52

Replying to @mkoeppe:

I'll add a feature DatabaseConwayPolynomials

Thanks. Unfortunately this doesn't help since the search patch is still defined in terms of CONWAY_POLYNOMIALS_DATA_DIR without a fallback.

I'm really strugging to understand the purpose of this change. Currently we have a long list of sane defaults in env.py which made everything work nicely out of the box. After this, if I understand correctly, are we expected to maintain a per-distro list of environment variables which essentially duplicates the list in env.py? This seems a big regression to me.

comment:53

Do you set SAGE_LOCAL in your distribution packaging?

Branch pushed to git repo; I updated commit sha1. New commits:

62c8990src/sage/features/databases.py: Only use CONWAY_POLYNOMIALS_DATA_DIR if it is set

Changed commit from e1eac9f to 62c8990

comment:55

Replying to @mkoeppe:

Do you set SAGE_LOCAL in your distribution packaging?

No, I don't set any env variable at all. Before this change, it was set to /usr in env.py via SAGE_LOCAL → SAGE_ENV → os.path.abspath(sys.prefix), which in turn made all other variables which depend on it point to the correct paths.

comment:56

Replying to @mkoeppe:

Replying to @dimpase:

I don't get sci-mathematics/lie on my Gentoo box, is it not yet in Gentoo?

No idea, I got this information from https://repology.org/project/lie/versions

It is not in the main tree, but it has been in the science overlay for a while. However, if you want one that will definitely work, you'll have to go with my sage-on-gentoo overlay. I updated it yesterday when I found that tarball has changed location.

comment:57

About sage_conf. I will want some small change to make it easier for distro, in a separate ticket of course, but my audience is here for now.

Currently sage_conf is a very customised python package in the sage tree that requires the top configure to be run for the variables to be setup to the proper values. Matthias encourage distro to write their own version of sage_conf. After giving it some thoughts, considering the way things works from source in Gentoo, writing a full blown python package that passes QA is too much of a hassle. This is not a place where we can just take a file and just install it in the right place(s) [OK, if you really insist we can abuse the system in such a way]. There are other issue like versioning that would need sorting. I don't want to maintain such a package.

But I still want to use sage_conf as it means that I don't to have patch env.py anymore. So, I write a small sage_conf.py file with my stuff, I install it alongside env.py and I change the import to be from sage.sage_conf instead of just sage_conf. What I would like to happen is for that option to be available out of the box in sage. Try sage_conf first and if it isn't present, try sage.sage_conf. The file is maintained alongside sage which means I don't have to worry about versioning amongst other things.

comment:58

Replying to @kiwifb:

I change the import to be from sage.sage_conf instead of just sage_conf. What I would like to happen is for that option to be available out of the box in sage. Try sage_conf first and if it isn't present, try sage.sage_conf.

That's terrible because then people who want to install the Sage library from source will not be able to use your distribution's sage_conf.

comment:59

Replying to @kiwifb:

Currently sage_conf is a very customised python package in the sage tree that requires the top configure to be run for the variables to be setup to the proper values. Matthias encourage distro to write their own version of sage_conf. After giving it some thoughts, considering the way things works from source in Gentoo, writing a full blown python package that passes QA is too much of a hassle.

Then just pass it off as a patched version of the upstream sage_conf package.

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

83d0b7esrc/sage/misc/compat.py: Do not fail if SAGE_LOCAL is unset
cd4ccd3src/sage/misc/cython.py: Do not fail if SAGE_LOCAL is not set
1a62da6src/sage/misc/dist.py: Fix typo
638182ebuild/bin/sage-starts, build/make/Makefile.in, src/sage/all.py: Remove use of sage-started.txt file
4a356a2src/sage/plot/plot3d/base.pyx: Remove unused imports, including SAGE_LOCAL
07a1a44src/sage/misc/citation.pyx: Do not fail if SAGE_LOCAL is unset; handle SAGE_VENV like SAGE_LOCAL
5645479Introduce LIE_INFO_DIR to point to the location of the info files of a LiE install and use it.
fa0520acorrect typo (dot to comma)
d7b9954sage.features.DatabaseConwayPolynomials: New, use it in sage.databases.conway
d88c884src/sage/features/databases.py: Only use CONWAY_POLYNOMIALS_DATA_DIR if it is set

Changed commit from 62c8990 to d88c884

comment:61

I have removed the initial commit, so the fallback is back. The other changes made here have independent value, so let's get them in.

Description changed:

--- 
+++ 
@@ -1,6 +1,6 @@
 (from [#31338 comment:13](https://github.com/sagemath/sage/issues/31338#comment:13))
 
-This default is no longer necessary because `sage_conf` is now a requirement; and should be removed.
+These changes are preparation for modularization and is also intended  to make downstream packaging easier.
 
 
 
comment:63

I honestly don't think they should use my distro's sage_conf. At least not in the current state. But it is an interesting prospect.

In the current state, I certainly don't want to use a patched version of your stuff. It has to become a proper pypi package first. Passing it off would be the same as writing one myself. I decided that I should draw some lines in the sand as to what I should do. I decided I should say "no" more often because I have already too much stuff to do, and in the current state that would clearly go there.


Last 10 new commits:

83d0b7esrc/sage/misc/compat.py: Do not fail if SAGE_LOCAL is unset
cd4ccd3src/sage/misc/cython.py: Do not fail if SAGE_LOCAL is not set
1a62da6src/sage/misc/dist.py: Fix typo
638182ebuild/bin/sage-starts, build/make/Makefile.in, src/sage/all.py: Remove use of sage-started.txt file
4a356a2src/sage/plot/plot3d/base.pyx: Remove unused imports, including SAGE_LOCAL
07a1a44src/sage/misc/citation.pyx: Do not fail if SAGE_LOCAL is unset; handle SAGE_VENV like SAGE_LOCAL
5645479Introduce LIE_INFO_DIR to point to the location of the info files of a LiE install and use it.
fa0520acorrect typo (dot to comma)
d7b9954sage.features.DatabaseConwayPolynomials: New, use it in sage.databases.conway
d88c884src/sage/features/databases.py: Only use CONWAY_POLYNOMIALS_DATA_DIR if it is set

Description changed:

--- 
+++ 
@@ -1,6 +1,6 @@
 (from [#31338 comment:13](https://github.com/sagemath/sage/issues/31338#comment:13))
 
-These changes are preparation for modularization and is also intended  to make downstream packaging easier.
+This default is no longer necessary because `sage_conf` is now a requirement; and should be removed.
 
 
 
comment:64

Restoring your changes.

Description changed:

--- 
+++ 
@@ -1,6 +1,6 @@
 (from [#31338 comment:13](https://github.com/sagemath/sage/issues/31338#comment:13))
 
-This default is no longer necessary because `sage_conf` is now a requirement; and should be removed.
+These changes are preparation for modularization and is also intended  to make downstream packaging easier.
 
 
 
comment:66

I am otherwise quite happy with the state of the ticket. I would have prefered the initial commit to stay in but we can do that later.

comment:68

Replying to @kiwifb:

Replying to @mkoeppe:

Replying to @dimpase:

I don't get sci-mathematics/lie on my Gentoo box, is it not yet in Gentoo?

No idea, I got this information from https://repology.org/project/lie/versions

It is not in the main tree, but it has been in the science overlay for a while. However, if you want one that will definitely work, you'll have to go with my sage-on-gentoo overlay. I updated it yesterday when I found that tarball has changed location.

I was able to install lie on Gentoo by doing

layman --add science
emerge --ask lie

Should we add

emerge --ask app-portage/layman
layman --add science

to whatever the current place for these per-system advice is (it was moved around...)

comment:69

Replying to @dimpase:

Replying to @kiwifb:

Replying to @mkoeppe:

Replying to @dimpase:

I don't get sci-mathematics/lie on my Gentoo box, is it not yet in Gentoo?

No idea, I got this information from https://repology.org/project/lie/versions

It is not in the main tree, but it has been in the science overlay for a while. However, if you want one that will definitely work, you'll have to go with my sage-on-gentoo overlay. I updated it yesterday when I found that tarball has changed location.

I was able to install lie on Gentoo by doing

layman --add science
emerge --ask lie

Should we add

emerge --ask app-portage/layman
layman --add science

to whatever the current place for these per-system advice is (it was moved around...)

Using layman is so outdated - even I moved on (last month :) ). Seriously, it is deprecated.

emerge eselect-repository 
eselect repository enable science 
emaint sync -r science 

or emerge --sync, all the repos managed that way will get pulled by emerge --sync or equivalent like eix-sync.

Reviewer: Dima Pasechnik

comment:70

OK

comment:71

Thanks!

comment:72

On OSX:

**********************************************************************
File "src/sage/misc/compat.py", line 97, in sage.misc.compat
Failed example:
    find_library('Singular')
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 718, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/sage/doctest/forker.py", line 1137, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.compat[1]>", line 1, in <module>
        find_library('Singular')
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/sage/misc/compat.py", line 102, in find_library
        result = _find_library(name)
      File "/Users/buildbot-sage/slave/sage_git/build/local/lib/python3.9/site-packages/sage/misc/compat.py", line 66, in _find_library
        os.environ['DYLD_LIBRARY_PATH'] = sage_local_lib
    NameError: name 'sage_local_lib' is not defined
**********************************************************************
comment:74

I see, we forgot to replace sage_local_lib which was defined as os.path.join(SAGE_LOCAL, 'lib') by expressions based on the new libdir in the darwin section. That was a bit careless of me not to notice.

Branch pushed to git repo; I updated commit sha1. New commits:

985af6esage.misc.compat._find_library [darwin]: Fix up

Changed commit from d88c884 to 985af6e

comment:76

Yes, looks like I fell asleep in the middle of rewriting this function.

comment:77

Replying to @mkoeppe:

Yes, looks like I fell asleep in the middle of rewriting this function.

The new code should do the trick. I must say that when we were working on this originally, it felt like you were in my time zone. So, I guess falling asleep would have been fair.