Shipyard configuration for running the covid19model on Azure batch / HPC
First an Azure batch account needs to be set-up. For this please follow the documentation from Microsoft, and make sure that you create a storage account during the setup. Create a batch account with Azure portal
Set the names and keys in the credentials.yaml.
You can either use Batch Shipyard locally or in the Azure cloud shell.
Batch Shipyard Installation Guide
Azure Cloud shell is the only method which was already tested for this setup.
Make sure to set the SHIPYARD_CONFIGDIR environment variable to the directory containing the YAML-files and this readme.
The pool of machines will be setup using shipyard.
shipyard pool add
When this command has finished you will also have created an SSH key in the files id_rsa_shipyard and id_rsa_shipyard.pub. Please safely guard and store those keys.
Once the pool setup is finished the job can be created using the following Shipyard Batch command:
Open the jobs.yaml file and set the do_not_run_until for both jobs to a date in the future.
shipyard jobs add --recreate
The same command can also be used if you want to perform any updates to the job.
Testing jobs can be added using this command
shipyard jobs add --jobs jobs-testing.yaml