nextstrain/ncov

feat: WDL overhaul for Dockstore and Terra

j23414 opened this issue · 0 comments

Description

In order to document and test the Terra WDL workflows, this issue outlines a somewhat major overhaul of WDL code.

To stop maintaining 3 separate GitHub branches for genbank_ingest, gisaid_ingest, and ncov; the code is refactored to using Dockstore subworkflows as discussed in this slack thread.

After some quick discussions with @huddlej , a proposed order of operations is listed below. Dockstore will only surface subworkflows on the master branch so documentation cannot "point to latest entry points" until the master .dockstore.yml is updated. Afterwards, new branches with the same .dockstore.yml will become available for future development.

Order of Operations

  • Surface the Dockstore subworkflows entries, which requires the following merge:
    • merge wdl/ingest_squash branch, which will merge the new .dockstore.yml and surface the subworkflows
    • delete the then redundant separate branches wdl/genbank_ingest and wdl/gisaid_ingest
  • Update wdl/docs branch to point to the surfaced Dockstore subworkflows
  • Tag a few beta testers to work through the wdl/docs and run subworkflows
  • Address any comments and suggestions
  • Merge wdl/docs (#999)
  • Migrate current users of wdl/ncov branch to master, and delete wdl/ncov branch

Any edits to WDL files and reStructuredText files should not affect the Snakemake workflows and should have limited side effects. Of course open to discussion/suggestions.