Add rules to the nextflu-private workflow to rename Auspice JSONs based on the build date and deploy the JSONs to the Nextstrain group
huddlej opened this issue · 0 comments
Before we can deploy newly built Auspice JSONs to the Nextstrain group, we need to rename them to a unique name that reflects the date they were built. The builds need to be renamed from names like h1n1pdm_2y_titers_ha.json
to flu_seasonal_YYYY-MM-DD_h1n1pdm_2y_titers_ha.json
where the YYYY-MM-DD
is the date the builds were run. We currently rename these files manually to include the date.
The nextstrain-public builds provide an example of custom rules to rename the JSONs prior to deployment. The new names for the JSONs get defined as a template string in the build configuration parameter called auspice_name
. The rename rules get invoked by the custom deploy rule which is the primary target of the workflow we run from GitHub Actions.
We should follow the same pattern for the nextflu-private builds, except we need to calculate the current date when the workflow runs. One approach could be to assign the current date to a global variable in the rename.smk
Snakemake file, define an auspice_name
for the builds with a non-wildcard placeholder for the date like "flu_seasonal_{{date}}_{lineage}_{{segment}}_{resolution}"
, and then pass the date value to the format
call on the filename strings (e.g., this one and this one).
TODO:
- Add custom rules to rename Auspice JSONs in nextflu-private profile
- Define
auspice_name
string for builds inprofiles/nextflu-private.yaml
with reference to the date - Format the Auspice JSON names with the date calculated from
datetime.date.today().strftime("%Y-%m-%d")
- Add custom rule to deploy renamed Auspice JSONs in the nextflu-private profile using the Nextstrain Group secrets
- Setup an AWS Cognito user with write access to the nextflu-private group (
nextflu-private-bot
) - Add nextflu-private username and password to seasonal-flu repo secrets for actions
- Update GitHub Action for nextflu-private builds to add two lines below the current environment variables passed to the CLI which specify
--env NEXTSTRAIN_USERNAME
and--env NEXTSTRAIN_PASSWORD
. These environment variables should already exist in the GitHub Action thanks to setup by the pathogen-repo-build action - Add
nextstrain login
command to the custom Snakemake deploy rule which will rely on credentials in environmental variables - Add a rule to the custom rename Snakefile to rename the forecast tip frequencies JSONs (e.g., from
h3n2_2y_ha_cell_fra_cTiter_x-ne_star_forecast-tip-frequencies.json
toflu_seasonal_YYYY-MM-DD_h3n2_2y_ha_cell_fra_cTiter_x-ne_star_forecast-tip-frequencies.json
or just store asauspice/forecasts/{build_name}/{model}.json
and copy toauspice/forecasts/{build_date}/{build_name}/{model}.json
)