drpatelh/nf-core-demultiplex

See the main fork of this repository here >>>

Introduction

nf-core/demultiplex is a bioinformatics pipeline used to demultiplex the raw data produced by next generation sequencing machines. At present, only Illumina sequencing data is supported.

The pipeline is built using Nextflow, a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker containers making installation trivial and results highly reproducible.

Pipeline summary

1. Reformatting the input sample sheet
   • Searches for [Data] tag
   • Splits 10X single cell samples into 10X, 10X-ATAC and 10X-DNA .csv files by searching in the sample sheet column DataAnalysisType for 10X-3prime, 10X-ATAC and 10X-CNV.
   • Outputs the results of needing to run specific processes in the pipeline (can be only 10X single cell samples, mix of 10X single cell with non single cell samples or all non single cell samples)
2. Checking the sample sheet for downstream error causing samples such as:
   • a mix of short and long indexes on the same lane
   • a mix of single and dual indexes on the same lane
3. Processes that only run if there are issues within the sample sheet found by the sample sheet check process (CONDITIONAL):
   1. Creates a new sample sheet with any samples that would cause an error removed and create a a txt file of a list of the removed problem samples
   2. Run bcl2fastq on the newly created sample sheet and output the Stats.json file
   3. Parsing the Stats.json file for the indexes that were in the problem samples list.
   4. Recheck newly made sample sheet for any errors or problem samples that did not match any indexes in the Stats.json file. If there is still an issue the pipeline will exit at this stage.
4. Single cell 10X sample processes (CONDITIONAL): NOTE: Must create CONFIG to point to CellRanger genome References
   1. Cell Ranger mkfastq runs only when 10X samples exist. This will run the process with CellRanger, CellRanger ATAC, and Cell Ranger DNA depending on which sample sheet has been created.
   2. Cell Ranger Count runs only when 10X samples exist. This will run the process with Cell Ranger Count, Cell Ranger ATAC Count, and Cell Ranger DNA CNVdepending on the output from Cell Ranger mkfastq. 10X reference genomes can be downloaded from the 10X site, a new config would have to be created to point to the location of these. Must add config to point Cell Ranger to genome references if used outside the Crick profile.
5. bcl2fastq (CONDITIONAL):
   1. Runs on either the original sample sheet that had no error prone samples or on the newly created sample sheet created from the extra steps.
   2. This is only run when there are samples left on the sample sheet after removing the single cell samples.
   3. The arguments passed in bcl2fastq are changeable parameters that can be set on the command line when initiating the pipeline. Takes into account if Index reads will be made into FastQ's as well
6. FastQC runs on the pooled fastq files from all the conditional processes.
7. FastQ Screen runs on the pooled results from all the conditional processes. Must have own fastq_screen config to direct to.
8. MultiQC runs on each projects FastQC results produced.
9. MultiQC_all runs on all FastQC results produced.

Samplesheet format

The input sample sheet must adhere to Illumina standards as outlined in the table below. Additional columns for DataAnalysisType and ReferenceGenome are required for the correct processing of 10X samples. The order of columns does not matter but the case of column name's does.

Lane	Sample_ID	index	index2	Sample_Project	ReferenceGenome	DataAnalysisType
1	ABC11A2	TCGATGTG	CTCGATGA	PM10000	Homo sapiens	Whole Exome
2	SAG100A10	SI-GA-C1		SC18100	Mus musculus	10X-3prime
3	CAP200A11	CTCGATGA		PM18200	Homo sapiens	Other

Quick Start

i. Install nextflow

ii. Install either Docker or Singularity for full pipeline reproducibility; see docs)

iii. Download the pipeline and test it on a minimal dataset with a single command

nextflow run nf-core/demultiplex -profile test,<docker/singularity/institute>

Please check nf-core/configs to see if a custom config file to run nf-core pipelines already exists for your Institute. If so, you can simply use -profile <institute> in your command. This will enable either docker or singularity and set the appropriate execution settings for your local compute environment.

iv. Start running your own analysis!

nextflow run nf-core/demultiplex -profile <docker/singularity/institute> --input samplesheet.csv  --run_dir /path/to/run/directory/

See usage docs for all of the available options when running the pipeline.

Documentation

The nf-core/demultiplex pipeline comes with documentation about the pipeline, found in the docs/ directory:

1. Installation
2. Pipeline configuration
   • Local installation
   • Adding your own system config
   • Reference genomes
3. Running the pipeline
4. Output and how to interpret the results
5. Troubleshooting

Credits

The nf-core/demultiplex pipeline was written by Chelsea Sawyer from The Bioinformatics & Biostatistics Group for use at The Francis Crick Institute, London.

Many thanks to others who have helped out along the way too, including (but not limited to): @ChristopherBarrington, @drpatelh, @danielecook, @escudem, @crickbabs

Contributions and Support

If you would like to contribute to this pipeline, please see the contributing guidelines.

For further information or help, don't hesitate to get in touch on Slack (you can join with this invite).

Citation

You can cite the nf-core publication as follows:

The nf-core framework for community-curated bioinformatics pipelines.

Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.

Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.
ReadCube: Full Access Link