Document how to schedule a workflow to run on specific branches
Opened this issue · 11 comments
I would like to be able to pass a specific branch to an OpenAstronomy workflow, so that I can use them to run a cron job against my release branches. Normally, to schedule a cron job on the non default branch one needs to specify:
- uses: actions/checkout@v3
with:
ref: non-default-branch
in the checkout phase.
Ideally, I would like to able to say, specify say cron-branches:
to the tox
workflow. When the workflow is triggered by a the cron
it launch the workflow on each of the cron-branches
rather than on the default branch for the workflow. That is if
jobs:
test:
uses: OpenAstronomy/github-actions-workflows/.github/workflows/tox.yml@v1
with:
posargs: '-n 4'
cron-branches:
- main
- release
envs: |
- linux: pep8
pytest: false
- macos: py310
- windows: py39-docs
libraries:
choco:
- graphviz
This workflow would launch 6 actions, one action for each env on each of the branches. Then if not triggered by cron
it would just run on the normal default branch.
Doing this would let me schedule my CI to run on my main
and release
branches so the CI gets run even if no PRs get opened or merged in a given time period.
Hi @WilliamJamieson, yes, scheduling it to run on different branches is useful, and you should be able to do this already. SunPy currently does this for the main
branch and two release branches. Runs of ci.yml
are triggered by scheduled_builds.yml
on a schedule. The benefit of this approach is that it runs the workflow file version that is located in the branch you want to test. For example, you may want py311
on main
even when release
is not ready for it, or you may want py38
on release
but no longer run it on main
.
Another benefit of maintaining a workflow file in each branch is that it keeps the status report easier to read. It may get difficult to quickly check the branch's test status if the jobs for all your branches are mixed into the same workflow run summary. This can get particularly complex if you have a chain of dependent jobs which must run in sequence (basic test -> complex tests -> doc tests).
Some important things to do to get it working:
ci.yml
needs to have theworkflow_dispatch
trigger listed- the
GITHUB_TOKEN
set inscheduled_builds.yml
needs to be a PAT token that you manually generate (because the defaultGITHUB_TOKEN
available in workflow runs does not have permission to call other workflows, to prevent infinite recursion)
Do you think that approach would work for you?
Thanks @ConorMacBride, I think this should work nicely for me. I did not realize this approach was possible. It might be a good idea to document an example of how to do this as part of these actions.
I do have some questions about the requirements for the PAT (I am basically looking to create as limited a token in scope as possible), as I haven't really had a need to use them before:
-
Namely, which if any OAuth scopes are needed to do this (classic tokens)?
- I think I only need to give it
workflows
permissions. Do I need even that?
- I think I only need to give it
-
Can this be done with a fine-grained token? If so, what level of permissions do I need to set?
- Is read-only enough for the
Actions
or will it needRead and write
. I there a different setting I need to set?
- Is read-only enough for the
Yeah, it would be good to properly document this. I think the fine grained token would be best because you can limit it to a single repository. I've not tried it but "actions: read and write" should be the minimum for the fine grained token. I can't remember the minimal settings for the old PAT. https://docs.github.com/en/rest/actions/workflows?apiVersion=2022-11-28#create-a-workflow-dispatch-event
Unfortunately, fine-grained tokens cannot be created with no expiration date, or at the very least one in the distant future (seems like the limit was 1year). This runs a bit contrary to the set-it and forget-it testing I want to schedule with the cron job (namely, release branches will see less and less activity as they age). For now I'll go with a classic token (unfortunately), or at least until github comes up with something for fine grained tokens that requires me to not think about them too much.
For classic tokens it looks like it needs a repo
OAuth scope based on the docs you linked to and me just trying different settings and seeing what worked.
@zanecodes, made a suggestion to me that may avoid the use of the PAT. Namely, turning the ci.yml
into a reusable workflow (I think it will need a workflow_call
in the on
section) and then modifying scheduled_builds.yml
to something like:
name: Scheduled builds
on:
# Allow manual runs through the web UI
workflow_dispatch:
schedule:
# ┌───────── minute (0 - 59)
# │ ┌───────── hour (0 - 23)
# │ │ ┌───────── day of the month (1 - 31)
# │ │ │ ┌───────── month (1 - 12 or JAN-DEC)
# │ │ │ │ ┌───────── day of the week (0 - 6 or SUN-SAT)
- cron: '0 7 * * *' # Every day at 07:00 UTC
jobs:
sunpy_main:
uses: sunpy/sunpy/.github/workflows/ci.yml@main
sunpy_4_0:
uses: sunpy/sunpy/.github/workflows/ci.yml@4.0
sunpy_4_1:
uses: sunpy/sunpy/.github/workflows/ci.yml@4.1
I think the issue then becomes that actions/checkout
in OpenAstronomy will checkout the code relative to the branch Scheduled builds
is triggered on (namely the default main
branch of the cron). In this case it might be useful to pass a ref:
down to the actions/checkout
in the OpenAstronomy workflow.
There are a few downsides to this, most notably being that this action won't affect any badges indicating the CI status on a given branch. So it is probably not that useful to me. The method you proposed earlier doesn't suffer from this issue.
Yeah, it would be good if the tokens would last for longer. Maybe there is some way to add a job to scheduled.yml
which opens an issue saying to update the token if the workflow dispatches fail (or if the token is close to expiring). At least then you would know.
I hadn't realised that the reusable workflows could now be nested. This used to not be possible but I see it now is! (https://docs.github.com/en/actions/using-workflows/reusing-workflows#nesting-reusable-workflows) Making sure it builds the correct branch is important and passing the ref:
through should work. ci.yml
would also need to only set the ref
if it was triggered by a workflow_call
event, so it can build PRs on the default ref. I think I will stick to the token method with SunPy as it seems to work well enough. We did have an issue with the token silently expiring recently and not noticing for a while, so maybe I'll look into notifications when it's close to expiring!
I think I will stick to the token method with SunPy as it seems to work well enough. We did have an issue with the token silently expiring recently and not noticing for a while, so maybe I'll look into notifications when it's close to expiring!
I think I will also use the same method in ASDF as well. Let me know if you figure out how to automatically notify when a token expires.
Apparently GitHub should email the creator of the token when its about to expire: https://github.blog/changelog/2021-07-26-expiration-options-for-personal-access-tokens/#:~:text=When%20using%20a%20personal%20access,as%20the%20expiration%20date%20approaches.
Ah, you're right. I now remember that the email got buried amongst 100s of other emails in the organisation email account
It would be nicer if it just raised an issue.
#142 added support for passing a specific branch to run a workflow, although I think this issue still raises a useful point, so I've made the title more specific.