This repository is an example of running the pipelines for converting between sequence data formats on Cromwell on Azure.
Learn more about using Azure for your Cromwell WDL workflows on our GitHub repo! - Cromwell on Azure.
This repository is a fork from the original and has all the required changes to run the WDL workflow on Cromwell on Azure.
Here, you can find the WDL files and an example inputs JSON files with links to data hosted on a public Azure Storage account. You can use the "datasettestinputs" storage account directly as a relative path, like in the inputs JSON file.
The bam-to-unmapped-bams.trigger.json, cram-to-bam.trigger.json, interleaved-fastq-to-paired-fastq.trigger.json and paired-fastq-to-unmapped-bam.trigger.json trigger files are ready to use. You can start the workflow on your instance of Cromwell on Azure, using these instructions.
Workflows for converting between sequence data formats
This script should convert a CRAM to SAM to BAM and outputs a BAM, BAM Index, and validation report. The reason this approach was chosen instead of converting CRAM to BAM directly using Samtools is because Samtools 1.3 produces incorrect bins due to an old version of htslib included in the package. Samtools versions 1.4 & 1.5 have an NM issue that causes them to not validate with Picard.
- Cram file
- Bam file and index
- Validation report
This WDL converts paired FASTQ to uBAM and adds read group information
- Pair-end sequencing data in FASTQ format (one file per orientation)
- The following metada descriptors per sample:
- readgroup
- sample_name
- library_name
- platform_unit
- run_date
- platform_name
- sequecing_center
- Unmapped BAM
This WDL converts BAM to unmapped BAMs
- BAM file
- Sorted Unmapped BAMs
- Text file listing the unmapped file paths (FOFN)
This WDL takes in a single interleaved(R1+R2) FASTQ file and separates it into separate R1 and R2 FASTQ (i.e. paired FASTQ) files. Paired FASTQ files are the input format for the tool that generates unmapped BAMs (the format used in most GATK processing and analysis tools).
- Interleaved Fastq file
- Separate R1 and R2 FASTQ files (i.e. paired FASTQ)
- GATK4 or later
- Samtools 1.3.1
- Picard 2.8.3
- Cromwell version support
- Successfully tested on v47
- Does not work on versions < v23 due to output syntax
- The provided JSON is a ready to use example JSON template of the workflow. Users are responsible for reviewing the GATK Tool and Tutorial Documentations to properly set the reference and resource variables.
- The following material is provided by the Data Science Platforum group at the Broad Institute. Please direct any questions or concerns to one of our forum sites : GATK or Terra.
Copyright Broad Institute, 2019 | BSD-3 This script is released under the WDL open source code license (BSD-3) (full license text at https://github.com/openwdl/wdl/blob/master/LICENSE). Note however that the programs it calls may be subject to different licenses. Users are responsible for checking that they are authorized to run all programs before running this script.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.