Genome Sketching

a review of sketching algorithms for genomic data

Overview

This repository accompanies my review of sketching algorithms for genomic data:

When The Levee Breaks: a practical guide to sketching algorithms for processing the flood of genomic data

The notebooks correspond to the sections in the paper:

Section 3: Sketching algorithms
- 3.1 Set similarity with MinHash
- 3.2 Set membership with Bloom filter
Section 4: Example workflows for genomics

I've also included a notebook to cover some basic concepts (e.g. k-mer decomposition). This notebook is called Background and we save ourselves some time later by importing the functions from this notebook into the other notebooks (e.g. for reading sequence data files).

If you've not used a Jupyter notebook before, just hit "shift + enter" to run a cell. Here is a good guide to get you started with Jupyter.

Running the workbooks

To run the workbooks interactively from your browser, use Binder:

Running the workbooks locally

If you want to run the workbooks locally, you'll need conda and jupyter.

Once you have these, follow these steps:

clone the repo

git clone https://github.com/will-rowe/genome-sketching

cd genome-sketching

create the environment

conda env create --file=binder/environment.yml

source activate genome-sketching

run the notebook server

jupyter notebook --notebook-dir=./notebooks

Notes

I'd like to thank Titus Brown and Luiz Irber, who not only got me onto Binder, but also inspired me to review the current sketching methods for genomics (thanks to their excellent sourmash documentation and examples of MinHash for genomics).

Thanks to Francesco Mosconi for showing how to use Travis CI for build testing the notebooks.