tox.ini, GitHub CI workflow - minor fixes, more homebrew configuration factors, improve docker interface

Question

tox.ini, GitHub CI workflow - minor fixes, more homebrew configuration factors, improve docker interface

mkoeppe opened this issue 6 years ago · 106 comments

mkoeppe commented 6 years ago

This makes the following changes to tox.ini:

tox -e local-homebrew-macos-standard now uses python3 from homebrew.

This attempts to reproduce the environment of https://groups.google.com/d/msg/sage-release/kU5M1QVuQQY/Zu10Q-CKAwAJ
tox -e local-homebrew-macos-standard-python3_xcode uses /usr/bin/python3 (was the old default)
tox -e local-homebrew-macos-standard-python3_pythonorg uses the Python 3.7 installation from the binary package from python.org

New toolchain factors: gcc_spkg, gcc_9.

Improved docker interface.

CC: @jhpalmieri @dimpase @orlitzky @kliem @vbraun

Component: porting

Author: Matthias Koeppe

Branch/Commit: b7bb418

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29417

Answer 1 · 2020-03-27T17:10:03.000Z

Dependencies: #29087, #29354

Answer 2 · 2020-03-27T17:11:19.000Z

Branch: u/mkoeppe/tox-ci-minor-fixes

Answer 3 · 2020-03-28T15:47:46.000Z

Changed dependencies from #29087, #29354 to #29087, #29354, #29418

Answer 4 · 2020-03-28T15:47:46.000Z

Last 10 new commits:

`fbb7393`	`Fixup packages`
`cfcd8f2`	`build/bin/sage-guess-package-system, build/bin/sage-print-system-package-command: Handle slackware too`
`136feeb`	`build/bin/sage-spkg: Show equivalent system packages for ALL distros`
`9939051`	`Merge branch 't/29354/tox_ini__add_slackware_environments' into t/29087/CI-gh-actions-tox-configure-build`
`4ac9e24`	`Reduce SAGE_NUM_THREADS to 3`
`1cb8f98`	`build/pkgs/curl/distros/slackware.txt: Add openldap-client`
`eee2877`	`build/pkgs/slackware.txt: Add ca-certificates`
`c1931f5`	`build/pkgs/curl/distros/slackware.txt: Add libssh2`
`3145467`	`Merge branch 't/29354/tox_ini__add_slackware_environments' into t/29087/CI-gh-actions-tox-configure-build`
`b259671`	`[tox-local-homebrew-macos] Repair make parallelization`

Answer 5 · 2020-03-28T15:47:46.000Z

Commit: b259671

Answer 6 · 2020-03-28T15:47:46.000Z

Author: Matthias Koeppe

Answer 7 · 2020-03-28T15:59:46.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`bd24641`	`build/bin/sage-download-file: Add option --no-check-certificate`
`e143939`	`build/bin/sage-spkg: Append to SAGE_DOWNLOAD_FILE_OPTIONS instead of overwriting it`
`710b52d`	`Merge branch 't/29418/sage_download_file__fix_certificate_problems_with_https_downloads_on_macos_python3' into t/29417/tox-ci-minor-fixes`
`b1489e7`	`tox.ini: Set SAGE_DOWNLOAD_FILE_OPTIONS=--no-check-certificate`

Answer 8 · 2020-03-28T15:59:46.000Z

Changed commit from b259671 to b1489e7

Answer 9 · 2020-03-29T20:27:18.000Z

Changed dependencies from #29087, #29354, #29418 to #29087, #29354, #29418, #29415

Answer 10 · 2020-03-30T19:44:52.000Z

Changed commit from b1489e7 to 846f14e

Answer 11 · 2020-03-30T19:44:52.000Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`354cf90`	`Use conda-install --yes --quiet`
`df609e0`	`Merge branch 't/29415/tox_ini__add_environment_local_conda' into t/29417/tox-ci-minor-fixes`
`cf27321`	`build/bin/sage-download-file: Add option --no-check-certificate`
`90ea00b`	`build/bin/sage-spkg: Append to SAGE_DOWNLOAD_FILE_OPTIONS instead of overwriting it`
`bccc2c2`	`Merge branch 't/29418/sage_download_file__fix_certificate_problems_with_https_downloads_on_macos_python3' into t/29417/tox-ci-minor-fixes`
`87b7351`	`tox.ini: Add configuration factors python3, macos-python3_xcode, macos-python3_pythonorg`
`329ea99`	`tox.yml: Test more homebrew configs`
`de1bce2`	`tox.yml: Add local-ubuntu-conda test`
`86d5c84`	`tox.ini: Add toolchain factors`
`846f14e`	`tox.yml: Test more Fedora, no conda`

Answer 12 · 2020-03-30T19:45:20.000Z

Changed dependencies from #29087, #29354, #29418, #29415 to #29087, #29418, #29415

Answer 13 · 2020-03-30T20:01:03.000Z

Changed commit from 846f14e to 9838fd7

Answer 14 · 2020-03-30T20:01:03.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`9838fd7`	`fixup`

Answer 15 · 2020-03-30T20:09:56.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`a3ccd46`	`build/pkgs/python3/distros/homebrew.txt: New`

Answer 16 · 2020-03-30T20:09:56.000Z

Changed commit from 9838fd7 to a3ccd46

Answer 17 · 2020-03-30T20:24:24.000Z

Description changed:

--- 
+++ 
@@ -1 +1,12 @@
+This makes the following changes to tox.ini:
+- `tox -e local-homebrew-macos-standard` now uses python3 from homebrew.
 
+   This attempts to reproduce the environment of https://groups.google.com/d/msg/sage-release/kU5M1QVuQQY/Zu10Q-CKAwAJ
+
+- `tox -e local-homebrew-macos-standard-python3_xcode` uses `/usr/bin/python3` (was the old default)
+
+- `tox -e local-homebrew-macos-standard-python3_pythonorg` uses the Python 3.7 installation from the binary package from python.org
+
+New toolchain factors: `gcc_spkg`, `gcc_9`.
+
+

Answer 18 · 2020-03-30T20:50:54.000Z

Changed dependencies from #29087, #29418, #29415 to #29087, #29418, #29415, #29401

Answer 19 · 2020-03-30T22:07:56.000Z

Changed commit from a3ccd46 to 2bc1243

Answer 20 · 2020-03-30T22:07:56.000Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`6153945`	`merge the sections on tox factors`
`ab11ebd`	`elide some details from terminal output`
`c7fcf91`	`change 'ls -l' to 'ls -la' and elide some details`
`e73bc8d`	`Elide more details, shorten directory names more`
`a31b2a0`	`Reviewer patch`
`b7b45fe`	`Reviewer patch`
`f7d2dc9`	`Remove old section head`
`99aca39`	`Reformat list, remove unfinished commented out section`
`2c53ae2`	`Merge branch 't/29401/add_documentation_of_tox_and_github_actions_workflow_to_developer_s_manual' into t/29417/tox-ci-minor-fixes`
`2bc1243`	`src/doc/en/developer/portability_testing.rst: Add documentation on new local-conda and local-homebrew environments`

Answer 21 · 2020-03-31T13:46:49.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`5dd7652`	`tox.ini: Add more linuxmint versions`
`c6cc486`	`tox.ini: IGNORE_MISSING_SYSTEM_PACKAGES=yes for old linuxmint versions`
`cd72da9`	`tox.ini: Add linuxmint-i386`
`7986ab3`	`tox.yml: Add more linuxmint`
`a54ab32`	`tox.ini: Remove linuxmint-20 (no docker image)`

Answer 22 · 2020-03-31T13:46:49.000Z

Changed commit from 2bc1243 to a54ab32

Answer 23 · 2020-04-02T16:13:46.000Z

Changed commit from a54ab32 to 4281c92

Answer 24 · 2020-04-02T16:13:46.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`3757212`	`tox.ini: Switch local-conda-forge to miniforge`
`a61c48e`	`tox.ini [local-conda]: Activate conda before calling conda install`
`8f58411`	`Merge branch 't/29415/tox_ini__add_environment_local_conda' into t/29417/tox-ci-minor-fixes`
`4281c92`	`Replace commands_pre by commands, to support tox < 3.4`

Answer 25 · 2020-04-02T16:39:38.000Z

Changed commit from 4281c92 to 49aeac3

Answer 26 · 2020-04-02T16:39:38.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`49aeac3`	`tox.ini: New factor nokegonly for homebrew`

Answer 27 · 2020-04-02T21:00:59.000Z

Changed commit from 49aeac3 to c7a7061

Answer 28 · 2020-04-02T21:00:59.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`c7a7061`	`tox.ini: Fix up 'nokegonly'`

Answer 29 · 2020-04-03T17:15:04.000Z

Changed commit from c7a7061 to 5477536

Answer 30 · 2020-04-03T17:15:04.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`5477536`	`tox.ini: Pass through DOCKER_BUILDKIT`

Answer 31 · 2020-04-03T19:19:02.000Z

Changed commit from 5477536 to 0b5a73a

Answer 32 · 2020-04-03T19:19:02.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`0b5a73a`	`tox.ini: Add factor python3_spkg`

Answer 33 · 2020-04-03T22:49:12.000Z

Changed commit from 0b5a73a to 85b3c51

Answer 34 · 2020-04-03T22:49:12.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`85b3c51`	`tox.ini: Set fedora <= 29 to IGNORE_MISSING_SYSTEM_PACKAGES=yes`

Answer 35 · 2020-04-04T21:02:22.000Z

Changed commit from 85b3c51 to 1414208

Answer 36 · 2020-04-04T21:02:22.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`1414208`	`Remove linuxmint-20, IGNORE_MISSING_SYSTEM_PACKAGES=yes for all linuxmint because of libbraiding`

Answer 37 · 2020-04-05T21:06:07.000Z

Changed commit from 1414208 to ba57c1c

Answer 38 · 2020-04-05T21:06:07.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`ba57c1c`	`tox.yml: Add debian-stretch`

Answer 39 · 2020-04-08T01:26:38.000Z

Changed commit from ba57c1c to 8f7e774

Answer 40 · 2020-04-08T01:26:38.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`544d886`	`build/bin/write-dockerfile.sh: Add 'FROM ... AS ...' so that docker build --target works`
`2845728`	`build/bin/write-dockerfile.sh: Add V=0 to default USE_MAKEFLAGS, to match 'tox local'`
`e655a03`	`tox.ini [docker]: Loop through DOCKER_TARGETS, tag intermediate images`
`09bdad0`	`tox.ini [docker]: Override default TARGETS_PRE and TARGETS using posargs, to match 'tox local' interface`
`4c8c72f`	`Fixup: Need to repeat ARG after FROM`
`8f7e774`	`src/doc/en/developer/portability_testing.rst: Document docker targets`

Answer 41 · 2020-04-08T01:30:14.000Z

Description changed:

--- 
+++ 
@@ -9,4 +9,5 @@
 
 New toolchain factors: `gcc_spkg`, `gcc_9`.
 
+Improved docker interface.

Answer 42 · 2020-04-08T13:02:40.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`03faba1`	`tox.ini: Fixup: Pass --always to git describe so it does not fail`

Answer 43 · 2020-04-08T13:02:40.000Z

Changed commit from 8f7e774 to 03faba1

Answer 44 · 2020-04-08T13:53:16.000Z

Changed commit from 03faba1 to 67dc11c

Answer 45 · 2020-04-08T13:53:16.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`67dc11c`	`build/bin/write-dockerfile.sh: Fixup for arch`

Answer 46 · 2020-04-08T19:33:44.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`b564e06`	`tox.ini [debian-stretch]: Set IGNORE_MISSING_SYSTEM_PACKAGES=yes`

Answer 47 · 2020-04-08T19:33:44.000Z

Changed commit from 67dc11c to b564e06

Answer 48 · 2020-04-09T18:35:51.000Z

Changed commit from b564e06 to 0c648dc

Answer 49 · 2020-04-09T18:35:51.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`0c648dc`	`tox.ini [ubuntu-bionic]: Set IGNORE_MISSING_SYSTEM_PACKAGES=yes`

Answer 50 · 2020-04-09T18:39:42.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`a2b1206`	`.github/workflows/tox.yml: Rename the jobs to improve display in [GitHub](../wiki/GitHub) Actions summary`

Answer 51 · 2020-04-09T18:39:42.000Z

Changed commit from 0c648dc to a2b1206

Answer 52 · 2020-04-09T19:20:23.000Z

comment:29

Do you know how to resolve this? Is this resolved already?

 Error processing tar file(exit status 1): write /sage/local/share/doc/sage/html/en/reference/modsym/sage/modular/modsym/relation_matrix_pyx.html: no space left on device
ERROR: InvocationError: '/usr/bin/docker build . -f /home/runner/work/sage-test-27122/sage-test-27122/.tox/docker-ubuntu-trusty-standard/Dockerfile --build-arg EXTRA_CONFIGURE_ARGS=   --build-arg BASE_IMAGE=ubuntu:trusty --build-arg USE_MAKEFLAGS=-k V=0 SAGE_NUM_THREADS=3 --build-arg TARGETS=build ptest'
___________________________________ summary ____________________________________
ERROR:   docker-ubuntu-trusty-standard: commands failed
##[error]Process completed with exit code 1.

This appears to be the reason that every single test is marked as failed, which is kind of annoying.

https://github.com/kliem/sage-test-27122/actions/runs/72102779

Answer 53 · 2020-04-09T19:24:55.000Z

comment:30

Replying to @kliem:

Do you know how to resolve this? Is this resolved already?

 Error processing tar file(exit status 1): write /sage/local/share/doc/sage/html/en/reference/modsym/sage/modular/modsym/relation_matrix_pyx.html: no space left on device
ERROR: InvocationError: '/usr/bin/docker build . -f /home/runner/work/sage-test-27122/sage-test-27122/.tox/docker-ubuntu-trusty-standard/Dockerfile --build-arg EXTRA_CONFIGURE_ARGS=   --build-arg BASE_IMAGE=ubuntu:trusty --build-arg USE_MAKEFLAGS=-k V=0 SAGE_NUM_THREADS=3 --build-arg TARGETS=build ptest'
___________________________________ summary ____________________________________
ERROR:   docker-ubuntu-trusty-standard: commands failed
##[error]Process completed with exit code 1.

Help with this would be most welcome!

I have the following browser tabs open, which should help with finding a solution. Essentially, one needs to reconfigure the docker daemon to use larger dm.basesize.

Docker - Using systemd to control the Docker daemon
https://success.docker.com/article/using-systemd-to-control-the-docker-daemon

"systemctl set-property docker" - Google Search
https://www.google.com/search?safe=off&biw=834&bih=836&sxsrf=ALeKk02AnzFOj0bspN6PGW6rKhDhhp3f4w%3A1586407757228&ei=TamOXtqxDYKWtQW5zr3YBA&q=%22systemctl+set-property+docker%22&oq=%22systemctl+set-property+docker%22&gs_lcp=CgZwc3ktYWIQAzIECAAQHjoECAAQRzoGCAAQFhAeSgkIFxIFMTItNjJKCAgYEgQxMi00UNOnAVi1rQFg2a8BaABwAngAgAF4iAHlAZIBAzAuMpgBAKABAaoBB2d3cy13aXo&sclient=psy-ab&ved=0ahUKEwiajNq2xdroAhUCS60KHTlnD0s4ChDh1QMIDA&uact=5

Docker - How to resolve Resource Temporarily Unavailable errors due to TaskMax setting
https://success.docker.com/article/how-to-reserve-resource-temporarily-unavailable-errors-due-to-tasksmax-setting

"systemctl status docker.service" dm.basesize - Google Search
https://www.google.com/search?safe=off&sxsrf=ALeKk01dBCgUutnZt6DU3HHzqaJyqDfA_w%3A1586408161041&ei=4aqOXpKKAtHKswXerrtY&q=%22systemctl+status+docker.service%22+dm.basesize&oq=%22systemctl+status+docker.service%22+dm.basesize&gs_lcp=CgZwc3ktYWIQAzIFCCEQoAE6BAgAEEc6BggAEBYQHjoFCAAQzQI6CAghEBYQHRAeSgoIFxIGMTItMTE0SgkIGBIFMTItMThQpp4DWN7FA2DvyANoBHADeACAAaIBiAHcD5IBBDAuMTWYAQCgAQGqAQdnd3Mtd2l6&sclient=psy-ab&ved=0ahUKEwjS_qD3xtroAhVR5awKHV7XDgsQ4dUDCAw&uact=5

Cannot Change Docker Size on Ubuntu 16.04 - DevOps Stack Exchange
https://devops.stackexchange.com/questions/4062/cannot-change-docker-size-on-ubuntu-16-04

uggds/vagrant-docker
https://github.com/uggds/vagrant-docker

Can't install Docker - Cloudera Community
https://community.cloudera.com/t5/Support-Questions/Can-t-install-Docker/td-p/205969

docker disk size - Google Search
https://www.google.com/search?safe=off&sxsrf=ALeKk02gDtblI6DspE7pP0chAR0f7lCuJQ%3A1586404959534&ei=X56OXs-YIMHisAWJ-rWwBg&q=docker+disk+size&oq=docker+disk+size&gs_lcp=CgZwc3ktYWIQAzICCAAyBggAEAcQHjIGCAAQBxAeMgYIABAHEB4yAggAMgYIABAHEB4yBggAEAcQHjIGCAAQBxAeMgYIABAHEB4yAggAOgQIABBHOgYIABAFEB46CAgAEAcQBRAeOgQIABANOggIABAHEAoQHjoICAAQCBAHEB5KEAgXEgwxMC0yMTJnMTg0ZzdKDAgYEggxMC0xZzJnMlDhZVjfdGDjeGgDcAJ4AIAB4gGIAfwNkgEFMC40LjWYAQCgAQGqAQdnd3Mtd2l6&sclient=psy-ab&ved=0ahUKEwiPxNSAu9roAhVBMawKHQl9DWYQ4dUDCAw&uact=5

elasticsearch - Disk size of a docker image - Stack Overflow
https://stackoverflow.com/questions/48751032/disk-size-of-a-docker-image

dockerd | Docker Documentation
https://docs.docker.com/engine/reference/commandline/dockerd/

virtual-environments/Ubuntu1804-README.md at master · actions/virtual-environments
https://github.com/actions/virtual-environments/blob/master/images/linux/Ubuntu1804-README.md

Docker Engine 18.09 release notes | Docker Documentation
https://docs.docker.com/engine/release-notes/18.09/

systemctl(1) — systemd — Debian jessie — Debian Manpages
https://manpages.debian.org/jessie/systemd/systemctl.1.en.html

Answer 54 · 2020-04-09T19:59:09.000Z

comment:31

This looks really complicated. I'm afraid this is way above what I know.

Does this help?

https://github.community/t5/GitHub-Actions/BUG-Strange-quot-No-space-left-on-device-quot-IOExceptions-on/m-p/47691#M6920

Answer 55 · 2020-04-09T21:17:53.000Z

comment:32

No worries - at least I was able to close this browser window now!

Answer 56 · 2020-04-10T12:56:58.000Z

comment:33

It appears that the solution in the link I provided, is fixing at least part of the problem.

I have put this

       - uses: actions/checkout@v2
+      - name: free disk space
+        run: |
+          sudo swapoff -a
+          sudo rm -f /swapfile
+          sudo apt clean
+          docker rmi $(docker image ls -aq)
+          df -h
       - name: Install test prerequisites

in the debian/ubuntu in .github/workflows/tox.yml and it seems to help. There are green checkmarks again. E.g. debian-jessie minimal passes https://github.com/kliem/sage-test-27122/actions/runs/75035611, which previously had that no space left on disk error.

Now many of minimal runs pass, but the standards show error messages as

The command '/bin/sh -c make SAGE_SPKG="sage-spkg -y -o" ${USE_MAKEFLAGS} ${TARGETS}' returned a non-zero code: 2
ERROR: InvocationError: '/bin/bash -c for docker_target in with-targets; do  docker build . -f /home/runner/work/sage-test-27122/sage-test-27122/.tox/docker-debian-jessie-standard/Dockerfile  --target $docker_target  --tag sage-docker-debian-jessie-standard-$docker_target:$(git describe --dirty --always)  --build-arg EXTRA_CONFIGURE_ARGS="  "  --build-arg BASE_IMAGE=debian:jessie  --build-arg TARGETS_PRE=sagelib-build-deps  --build-arg TARGETS=build  --build-arg USE_MAKEFLAGS="-k V=0 SAGE_NUM_THREADS=3" --build-arg TARGETS="build ptest"; done'

Answer 57 · 2020-04-10T12:59:02.000Z

comment:34

In the link they also cleanup old artifacts

    - name: cleanup old action artifacts
      run: .github/workflows/purge_artifacts.sh ${{ secrets.REPO_ACCESS_TOKEN }}

I don't know what this is for. But maybe that is also helpful.

Answer 58 · 2020-04-10T13:18:10.000Z

comment:35

That is probably the expected behavior when not all tests pass.

Replying to @kliem:

The command '/bin/sh -c make SAGE_SPKG="sage-spkg -y -o" ${USE_MAKEFLAGS} ${TARGETS}' returned a non-zero code: 2
ERROR: InvocationError: '/bin/bash -c for docker_target in with-targets; do  docker build . -f /home/runner/work/sage-test-27122/sage-test-27122/.tox/docker-debian-jessie-standard/Dockerfile  --target $docker_target  --tag sage-docker-debian-jessie-standard-$docker_target:$(git describe --dirty --always)  --build-arg EXTRA_CONFIGURE_ARGS="  "  --build-arg BASE_IMAGE=debian:jessie  --build-arg TARGETS_PRE=sagelib-build-deps  --build-arg TARGETS=build  --build-arg USE_MAKEFLAGS="-k V=0 SAGE_NUM_THREADS=3" --build-arg TARGETS="build ptest"; done'

Answer 59 · 2020-04-10T15:13:07.000Z

comment:36

Replying to @kliem:

That is probably the expected behavior when not all tests pass.

Replying to @kliem:

The command '/bin/sh -c make SAGE_SPKG="sage-spkg -y -o" ${USE_MAKEFLAGS} ${TARGETS}' returned a non-zero code: 2
ERROR: InvocationError: '/bin/bash -c for docker_target in with-targets; do  docker build . -f /home/runner/work/sage-test-27122/sage-test-27122/.tox/docker-debian-jessie-standard/Dockerfile  --target $docker_target  --tag sage-docker-debian-jessie-standard-$docker_target:$(git describe --dirty --always)  --build-arg EXTRA_CONFIGURE_ARGS="  "  --build-arg BASE_IMAGE=debian:jessie  --build-arg TARGETS_PRE=sagelib-build-deps  --build-arg TARGETS=build  --build-arg USE_MAKEFLAGS="-k V=0 SAGE_NUM_THREADS=3" --build-arg TARGETS="build ptest"; done'

Yes, this is expected behavior.

But it would perhaps be better to change it so that the docker build succeeds if there are only testsuite failures

Answer 60 · 2020-04-10T17:04:52.000Z

comment:37

Replying to @mkoeppe:

But it would perhaps be better to change it so that the docker build succeeds if there are only testsuite failures

Working on a solution for this.

Answer 61 · 2020-04-10T18:20:16.000Z

comment:38

I think the above thing with the disk space is more important.

I like the green check marks, because then I can just trust that everything worked fine. However,
it would still be nice, if one can quickly access which tests failed. But maybe that is non-trivial and then that's just the way it is.

Also I still hope that very soon we will have most of the runs clean either way.

Answer 62 · 2020-04-10T18:25:42.000Z

comment:39

Replying to @kliem:

I think the above thing with the disk space is more important.

Yes, I'm trying out your patch

Answer 63 · 2020-04-10T19:09:36.000Z

comment:40

Maybe one could have failing doctests communicated as neutral runs (according to github this is also a possible state). I don't know how much work it would be to accomplish this, but it would be really nice. Than one knows at the first glance what's going on.

Answer 64 · 2020-04-10T19:18:47.000Z

comment:41

I don't think it is supported anymore https://help.github.com/en/actions/building-actions/setting-exit-codes-for-actions

I rather have failed tests as failure so that one has to check the details. Otherwise we might overlook critical failing tests.

Answer 65 · 2020-04-11T00:14:07.000Z

comment:42

Replying to @kliem:

I rather have failed tests as failure so that one has to check the details. Otherwise we might overlook critical failing tests.

In the branch that I'm testing right now this is conveniently configurable

Answer 66 · 2020-04-11T16:39:42.000Z

comment:43

A preview at https://github.com/mkoeppe/sage/actions/runs/75739694

Answer 67 · 2020-04-11T16:53:04.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`856e35c`	`tox.ini, tox.yml: Do not fail if errors occur while making $TARGETS_OPTIONAL (default: ptest)`
`484e01d`	`naive guess to clean disk space`
`94f64a8`	`tox fixup`
`e389763`	`tox fixup`
`ad359c4`	`.github/workflows/scan-logs.sh: Fixup regex`
`e45bc0e`	`.github/workflows/scan-logs.sh: Rewording for better display`

Answer 68 · 2020-04-11T16:53:04.000Z

Changed commit from a2b1206 to e45bc0e

Answer 69 · 2020-04-12T05:13:03.000Z

Changed commit from e45bc0e to 3a02031

Answer 70 · 2020-04-12T05:13:03.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`19edf1d`	`tox.ini: Store successfully built docker tags in {toxenv}/Dockertags`
`3a02031`	`tox.yml: Retrieve logs from successful builds via {toxenv}/Dockertags`

Answer 71 · 2020-04-12T07:26:07.000Z

comment:46

Running at https://github.com/mkoeppe/sage/actions/runs/76368041

Answer 72 · 2020-04-12T17:38:40.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`06c28fe`	`.github/workflows/tox.yml: Pass TARGETS to tox so that errors in doc-html count as failures`
`5ce717b`	`.github/workflows/scan-logs.sh: Fixup regexp for scanning ptest.log`
`53e2a83`	`.github/workflows/scan-logs.sh: Also catch errors of the form ImportError`

Answer 73 · 2020-04-12T17:38:40.000Z

Changed commit from 3a02031 to 53e2a83

Answer 74 · 2020-04-13T00:04:06.000Z

comment:50

This now seems to work very well. https://github.com/mkoeppe/sage/actions/runs/76629492

Failing TARGETS_OPTIONAL (default = ptest) do not abort the build. These show as green checkmarks in the left pane, but for all failing doctest logs (both packages and sagelib ptest) also a yellow "Check warning" is displayed as annotation in the right pane.

By setting TARGETS_OPTIONAL to include names of optional packages, we can also test these in the same way.

Answer 75 · 2020-04-13T05:32:06.000Z

Changed dependencies from #29087, #29418, #29415, #29401 to none

Answer 76 · 2020-04-13T05:33:20.000Z

Changed commit from 53e2a83 to 70d472e

Answer 77 · 2020-04-13T05:33:20.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`70d472e`	`tox.ini: Quoting fix`

Answer 78 · 2020-04-13T05:34:30.000Z

Changed commit from 70d472e to d94969c

Answer 79 · 2020-04-13T05:34:30.000Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

`6fc9a18`	`tox fixup`
`4d2366b`	`tox fixup`
`828394b`	`.github/workflows/scan-logs.sh: Fixup regex`
`a39ca99`	`.github/workflows/scan-logs.sh: Rewording for better display`
`0a02589`	`tox.ini: Store successfully built docker tags in {toxenv}/Dockertags`
`d209e8e`	`tox.yml: Retrieve logs from successful builds via {toxenv}/Dockertags`
`f1858e7`	`.github/workflows/tox.yml: Pass TARGETS to tox so that errors in doc-html count as failures`
`f9aca59`	`.github/workflows/scan-logs.sh: Fixup regexp for scanning ptest.log`
`b32e1bf`	`.github/workflows/scan-logs.sh: Also catch errors of the form ImportError`
`d94969c`	`tox.ini: Quoting fix`

Answer 80 · 2020-04-13T05:34:56.000Z

comment:54

Rebased on 9.1.rc0

Answer 81 · 2020-04-13T15:44:00.000Z

comment:55

Test for 9.1.rc0 running at https://github.com/mkoeppe/sage/actions/runs/76923542

Answer 82 · 2020-04-13T17:20:29.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`faf0ed6`	`tox.ini: Another quoting fix`

Answer 83 · 2020-04-13T17:20:29.000Z

Changed commit from d94969c to faf0ed6

Answer 84 · 2020-04-15T01:24:07.000Z

Changed commit from faf0ed6 to aed3fb8

Answer 85 · 2020-04-15T01:24:07.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`aed3fb8`	`src/doc/en/developer/portability_testing.rst: Explain how to read [GitHub](../wiki/GitHub) Actions page`

Answer 86 · 2020-04-15T01:24:39.000Z

comment:58

Can we finalize the review of this one please to get it into 9.1?

Answer 87 · 2020-04-15T01:55:58.000Z

comment:59

running GH Actions tests on this here: dimpase#3

Answer 88 · 2020-04-15T17:44:37.000Z

comment:60

Replying to @dimpase:

running GH Actions tests on this here: dimpase#3

I see

/Users/runner/runners/2.168.0/work/sage/sage/build/bin/sage-venv "/Users/runner/runners/2.168.0/work/sage/sage/.tox/local-homebrew-macos-python3_pythonorg-standard/local"
2020-04-15T16:31:11.9205990Z /bin/sh: /Library/Frameworks/Python.framework/Versions/3.7/bin/python3: No such file or directory
2020-04-15T16:31:11.9206560Z make[1]: *** [/Users/runner/runners/2.168.0/work/sage/sage/.tox/local-homebrew-macos-python3_pythonorg-standard/local/pyvenv.cfg] Error 127
2020-04-15T16:31:11.9207060Z make[1]: Target `doc-html' not remade because of errors.

there. Isn't this ticked supposed to be fixing this?

Answer 89 · 2020-04-15T18:17:49.000Z

comment:61

Thanks for catching this. Turns out I forgot to install the python.org python3 in tox.yml.
(This cannot be done in tox.ini because it is not relocatable, see comment there)

Answer 90 · 2020-04-15T18:38:32.000Z

Changed commit from aed3fb8 to d39d627

Answer 91 · 2020-04-15T18:38:32.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`d39d627`	`tox.yml [python3_xcode]: Install the package`

Answer 92 · 2020-04-15T18:42:51.000Z

Changed commit from d39d627 to ba0c266

Answer 93 · 2020-04-15T18:42:51.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`ba0c266`	`fixup`

Answer 94 · 2020-04-15T21:10:16.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`9060d23`	`fixup`

Answer 95 · 2020-04-15T21:10:16.000Z

Changed commit from ba0c266 to 9060d23

Answer 96 · 2020-04-16T02:13:22.000Z

Changed commit from 9060d23 to b7bb418

Answer 97 · 2020-04-16T02:13:22.000Z

Branch pushed to git repo; I updated commit sha1. New commits:

`b7bb418`	`tox.yml [macos-python3_pythonorg]: Use sudo with installer`

Answer 98 · 2020-04-16T08:02:47.000Z

comment:66

next try here: dimpase#4

Answer 99 · 2020-04-17T15:55:26.000Z

comment:67

homebrew-macos-python3_pythonorg, standard (https://github.com/dimpase/sage/runs/591414002) crashes at [dochtml] stage - not sure whether one can get complete logs/dochtml.log file from GitHub run above

Answer 100 · 2020-04-17T16:03:33.000Z

comment:68

I presume that various errors on local-macos standard runs are to be debugged individually.

e.g. on https://github.com/dimpase/sage/runs/591413919 (homebrew-macos-python3_xcode, minimal) I see ##[error] ==== ERROR IN LOG FILE artifacts/logs-commit-8bb28ec8ffd6869c6b86e3f5224333decb049d09-tox-local-homebrew-macos-python3_xcode-minimal/log/pkgs/gc-7.6.4.p0.log ==== and
Warning: Error testing package gc-7.6.4.p0 (ignored)