sagemath/sage

Add more homebrew system package information, including packages without spkg-configure.m4

mkoeppe opened this issue · 38 comments

mkoeppe commented

Along the same lines as #30859 for Debian,
this is for several purposes:

  • test proper isolation of our build (i.e., the presence of unused system packages does not break our build) using tox -e ...-maximal
  • reveal errors such as #30745 (homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local), #31132
  • test sagelib without Sage-the-distribution (similar to #28745/#30845 for conda)
  • prepare for tickets that will add spkg-configure.m4

(see also #29146)

(from #30371)

Related:

  • #27330: Meta-ticket: Use all system packages we can
  • #30504: Add MacPorts package information

This ticket does not cover Homebrew packages available via taps.

Interested users can install GAP, Macaulay2 and Polymake as follows:

$ brew tap gap-system/gap
$ brew install gap

$ brew tap macaulay2/tap
$ brew install macaulay2

$ brew tap apaffenholz/polymake
$ brew install polymake

Depends on #29124

CC: @tobiasdiez @slel @dimpase @jhpalmieri

Component: build: configure

Keywords: homebrew, spkg-configure.m4, macos

Author: Samuel Lelièvre, Matthias Koeppe

Branch/Commit: c2b36c7

Reviewer: Matthias Koeppe, Samuel Lelièvre

Issue created by migration from https://trac.sagemath.org/ticket/30861

mkoeppe commented

Description changed:

--- 
+++ 
@@ -5,5 +5,14 @@
 - Testing sagelib without Sage-the-distribution (similar to #28745/#30845 for conda)
 - Preparation for tickets that will add `spkg-configure.m4`
 
+In contrast to #30589, this needs additional refactoring work, to address this FIXME in `tox.ini`:
+
+```
+    # -standard and -maximal are for now the same. This should be fixed when refactoring write-dockerfile.
+    homebrew-standard:   bash -c 'PACKAGES=`sed "s/#.*//;" build/pkgs/*/distros/homebrew.txt`; {env:HOMEBREW}/bin/brew install $PACKAGES; {env:HOMEBREW}/bin/brew upgrade $PACKAGES'
+    homebrew-maximal:    bash -c 'PACKAGES=`sed "s/#.*//;" build/pkgs/*/distros/homebrew.txt`; {env:HOMEBREW}/bin/brew install $PACKAGES; {env:HOMEBREW}/bin/brew upgrade $PACKAGES'
+```
+
+
 (from #30371)
mkoeppe commented

Description changed:

--- 
+++ 
@@ -12,6 +12,7 @@
     homebrew-standard:   bash -c 'PACKAGES=`sed "s/#.*//;" build/pkgs/*/distros/homebrew.txt`; {env:HOMEBREW}/bin/brew install $PACKAGES; {env:HOMEBREW}/bin/brew upgrade $PACKAGES'
     homebrew-maximal:    bash -c 'PACKAGES=`sed "s/#.*//;" build/pkgs/*/distros/homebrew.txt`; {env:HOMEBREW}/bin/brew install $PACKAGES; {env:HOMEBREW}/bin/brew upgrade $PACKAGES'
 ```
+(see also #29146)
 
 
 (from #30371)
slel commented

Changed keywords from none to homebrew, spkg-configure.m4

slel commented

Description changed:

--- 
+++ 
@@ -1,11 +1,11 @@
-Along the same lines as #30859 for debian, 
+Along the same lines as #30859 for Debian, 
 this is for several purposes:
 
-- Test proper isolation of our build (i.e., the presence of unused system packages do not break our build) using `tox -e ...-maximal` - to reveal errors such as #30745 (homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local)
-- Testing sagelib without Sage-the-distribution (similar to #28745/#30845 for conda)
-- Preparation for tickets that will add `spkg-configure.m4`
+- Test proper isolation of our build (i.e., the presence of unused system packages does not break our build) using `tox -e ...-maximal` - Reveal errors such as #30745 (homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local)
+- Test sagelib without Sage-the-distribution (similar to #28745/#30845 for conda)
+- Prepare for tickets that will add `spkg-configure.m4`
 
-In contrast to #30589, this needs additional refactoring work, to address this FIXME in `tox.ini`:
+In contrast to #30859, this needs additional refactoring work, to address this FIXME in `tox.ini`:
 
 ```
     # -standard and -maximal are for now the same. This should be fixed when refactoring write-dockerfile.
slel commented

Description changed:

--- 
+++ 
@@ -1,9 +1,10 @@
 Along the same lines as #30859 for Debian, 
 this is for several purposes:
 
-- Test proper isolation of our build (i.e., the presence of unused system packages does not break our build) using `tox -e ...-maximal` - Reveal errors such as #30745 (homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local)
-- Test sagelib without Sage-the-distribution (similar to #28745/#30845 for conda)
-- Prepare for tickets that will add `spkg-configure.m4`
+- test proper isolation of our build (i.e., the presence of unused system packages does not break our build) using `tox -e ...-maximal`
+- reveal errors such as #30745 (homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local)
+- test sagelib without Sage-the-distribution (similar to #28745/#30845 for conda)
+- prepare for tickets that will add `spkg-configure.m4`
 
 In contrast to #30859, this needs additional refactoring work, to address this FIXME in `tox.ini`:
 
@@ -12,8 +13,8 @@
     homebrew-standard:   bash -c 'PACKAGES=`sed "s/#.*//;" build/pkgs/*/distros/homebrew.txt`; {env:HOMEBREW}/bin/brew install $PACKAGES; {env:HOMEBREW}/bin/brew upgrade $PACKAGES'
     homebrew-maximal:    bash -c 'PACKAGES=`sed "s/#.*//;" build/pkgs/*/distros/homebrew.txt`; {env:HOMEBREW}/bin/brew install $PACKAGES; {env:HOMEBREW}/bin/brew upgrade $PACKAGES'
 ```
+
 (see also #29146)
-
 
 (from #30371)
slel commented

Changed keywords from homebrew, spkg-configure.m4 to homebrew, spkg-configure.m4, macos

mkoeppe commented

Dependencies: #30865

mkoeppe commented

Description changed:

--- 
+++ 
@@ -6,7 +6,7 @@
 - test sagelib without Sage-the-distribution (similar to #28745/#30845 for conda)
 - prepare for tickets that will add `spkg-configure.m4`
 
-In contrast to #30859, this needs additional refactoring work, to address this FIXME in `tox.ini`:
+In contrast to #30859, this needs additional refactoring work (#30865), to address this FIXME in `tox.ini`:
 
 ```
     # -standard and -maximal are for now the same. This should be fixed when refactoring write-dockerfile.
mkoeppe commented

Changed dependencies from #30865 to #30865, #30951

mkoeppe commented

Description changed:

--- 
+++ 
@@ -6,7 +6,7 @@
 - test sagelib without Sage-the-distribution (similar to #28745/#30845 for conda)
 - prepare for tickets that will add `spkg-configure.m4`
 
-In contrast to #30859, this needs additional refactoring work (#30865), to address this FIXME in `tox.ini`:
+In contrast to #30859, this needs additional refactoring work (#30865, #30951), to address this FIXME in `tox.ini`:
 
 ```
     # -standard and -maximal are for now the same. This should be fixed when refactoring write-dockerfile.
mkoeppe commented

Changed dependencies from #30865, #30951 to #30865, #30951, #30745

mkoeppe commented

Description changed:

--- 
+++ 
@@ -6,7 +6,7 @@
 - test sagelib without Sage-the-distribution (similar to #28745/#30845 for conda)
 - prepare for tickets that will add `spkg-configure.m4`
 
-In contrast to #30859, this needs additional refactoring work (#30865, #30951), to address this FIXME in `tox.ini`:
+In contrast to #30859, this needs additional refactoring work (#30865, #30951, #29124), to address this FIXME in `tox.ini`:
 
 ```
     # -standard and -maximal are for now the same. This should be fixed when refactoring write-dockerfile.
mkoeppe commented

Changed dependencies from #30865, #30951, #30745 to #30865, #30951, #30745, #29124

mkoeppe commented
comment:8

The preparations are done in #29124. Now the system packages can be added

mkoeppe commented

Changed dependencies from #30865, #30951, #30745, #29124 to #29124

mkoeppe commented

Description changed:

--- 
+++ 
@@ -2,17 +2,9 @@
 this is for several purposes:
 
 - test proper isolation of our build (i.e., the presence of unused system packages does not break our build) using `tox -e ...-maximal`
-- reveal errors such as #30745 (homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local)
+- reveal errors such as #30745 (homebrew: Errors when homebrew's ntl, flint, or ecl are installed in /usr/local), #31132
 - test sagelib without Sage-the-distribution (similar to #28745/#30845 for conda)
 - prepare for tickets that will add `spkg-configure.m4`
-
-In contrast to #30859, this needs additional refactoring work (#30865, #30951, #29124), to address this FIXME in `tox.ini`:
-
-```
-    # -standard and -maximal are for now the same. This should be fixed when refactoring write-dockerfile.
-    homebrew-standard:   bash -c 'PACKAGES=`sed "s/#.*//;" build/pkgs/*/distros/homebrew.txt`; {env:HOMEBREW}/bin/brew install $PACKAGES; {env:HOMEBREW}/bin/brew upgrade $PACKAGES'
-    homebrew-maximal:    bash -c 'PACKAGES=`sed "s/#.*//;" build/pkgs/*/distros/homebrew.txt`; {env:HOMEBREW}/bin/brew install $PACKAGES; {env:HOMEBREW}/bin/brew upgrade $PACKAGES'
-```
 
 (see also #29146)
slel commented

Author: Samuel Lelièvre

slel commented
comment:10

Ready for review.

New commits:

dc64c6230861: Add Homebrew package info
slel commented

Description changed:

--- 
+++ 
@@ -10,3 +10,6 @@
 
 (from #30371)
 
+Related:
+
+- #30504: Add MacPorts package information
slel commented

Commit: dc64c62

slel commented

Branch: public/30861

slel commented
comment:11

Also available from Macaulay2 taps: fflas-ffpack, frobby, gfan, givaro.

However they break building Sage, see

Also available from GAP tap: gap

slel commented

Description changed:

--- 
+++ 
@@ -12,4 +12,5 @@
 
 Related:
 
+#27330: Meta-ticket: spkg-configure: Try to use as many system packages as possible
 - #30504: Add MacPorts package information
mkoeppe commented
comment:13

We don't really have a mechanism yet to ask users to use a tap - I would need that for polymake too - #31527

slel commented

Description changed:

--- 
+++ 
@@ -12,5 +12,21 @@
 
 Related:
 
-#27330: Meta-ticket: spkg-configure: Try to use as many system packages as possible
+- #27330: Meta-ticket: Use all system packages we can
 - #30504: Add MacPorts package information
+
+This ticket does not cover Homebrew packages available via taps.
+
+Interested users can install GAP, Macaulay2 and Polymake as follows:
+
+```
+$ brew tap gap-system/gap
+$ brew install gap
+
+$ brew tap macaulay2/tap
+$ brew install macaulay2
+
+$ brew tap apaffenholz/polymake
+$ brew install polymake
+```
+
slel commented
comment:15

The homebrew-maximal test environment could install extra packages from taps anyway.

mkoeppe commented
comment:16

Replying to @slel:

The homebrew-maximal test environment could install extra packages from taps anyway.

Well, it's easy to implement in build/bin/sage-print-system-package-command - in order to give both advice for users and set up the -maximal environment. I think I'll implement this in the polymake ticket.

mkoeppe commented
comment:17

The mactex bottle is giving me an error when I do tox -e local-homebrew-macos-maximal -- config.status.

==> Downloading http://mirror.ctan.org/systems/mac/mactex/mactex-20200407.pkg
==> Downloading from https://ctan.math.illinois.edu/systems/mac/mactex/mactex-20200407.pkg
######################################################################## 100.0%
==> Installing dependencies: jpeg, libtiff, ghostscript
==> Downloading https://homebrew.bintray.com/bottles/jpeg-9d.catalina.bottle.tar.gz
Already downloaded: /Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/.tox/local-homebrew-macos-maximal/Library/Caches/Homebrew/downloads/9ed6b08679bb12bb63055c727e5cdcb7f0813e6a89a66ee35a2b2b83d1c30ac9--jpeg-9d.catalina.bottle.tar.gz
==> Installing jpeg
Error: No such file or directory @ dir_chdir - /Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/.tox/local-homebrew-macos-maximal/homebrew/Cellar

Let's remove build/pkgs/texlive/distros/homebrew.txt and revisit this in #31529.

sagetrac-git commented

Changed commit from dc64c62 to 72d6b13

sagetrac-git commented

Branch pushed to git repo; I updated commit sha1. New commits:

4eb1ba7build/pkgs/texlive/distros/homebrew.txt: Remove to avoid installation errors
72d6b13build/pkgs/singular/distros/homebrew.txt: Disable to avoid installation errors
mkoeppe commented
comment:19

With these changes, it goes through and reports:


    notice: the following SPKGs did not find equivalent system packages:

        _recommended brial cddlib cliquer coxeter3 eclib ecm fflas_ffpack flintqs gf2x gfan giac givaro gp2c iml lcalc libbraiding libhomfly libsemigroups lrcalc m4ri m4rie palp pari pari_elldata pari_galdata pari_galpol pari_nftables pari_seadata pari_seadata_small perl_cpan_polymake_prereq perl_term_readline_gnu planarity rw symmetrica sympow tachyon zn_poly
        
checking for the package system in use... homebrew
configure:

    hint: installing the following system packages, if not
    already present, is recommended and may avoid having to
    build them (though some may have to be built anyway):

      $ brew install ffmpeg imagemagick texinfo cddlib
sagetrac-git commented

Branch pushed to git repo; I updated commit sha1. New commits:

c2b36c7build/pkgs/cddlib/distros/homebrew.txt: Disable cddlib for now (waiting for #29413)
sagetrac-git commented

Changed commit from 72d6b13 to c2b36c7

mkoeppe commented

Reviewer: Matthias Koeppe

slel commented
comment:22

Thanks!

slel commented

Changed author from Samuel Lelièvre to Samuel Lelièvre, Matthias Koeppe

slel commented

Changed reviewer from Matthias Koeppe to Matthias Koeppe, Samuel Lelièvre

vbraun commented

Changed branch from public/30861 to c2b36c7