Rabix is an open source implementation of the specification being developed on the Common Workflow Language mailing list.
CWL is an informal task force consisting of people from various organizations that have an interest in portability of bioinformatics workflows. The goal is to specify a way to describe bioinformatics tools and workflows that is powerful, easy to use and allows for portability of tools/workflows and reproducibility of runs.
Version 0.7 is approaching full compatibility to draft2 specification. To play with describing tools and making workflows visit rabix.org.
This repo includes a local python executor and some utilities for building docker images.
We've pre-installed rabix and dependencies on a VirtualBox machine image. If you install VirtualBox and Vagrant, you can run the machine using the vagrantfile:
wget https://s3.amazonaws.com/rabix/Vagrantfile
vagrant up && vagrant ssh
# Example BWA run with local files
sudo pip install -U rabix
cd rabix/examples
rabix https://s3.amazonaws.com/rabix/rabix-test/bwa-mem.json -- --reads test-data/example_human_Illumina.pe_1.fastq --reads test-data/example_human_Illumina.pe_2.fastq --reference test-data/chr20.fa
If you launch an instance on AWS in the us-east region, you can use the public AMI ami-60644508 which has rabix and dependencies installed. SSH to the instance using "rabix" as username and password.
To run the BWA example:
sudo pip install -U rabix
cd examples
rabix https://s3.amazonaws.com/rabix/rabix-test/bwa-mem.json -- --reads test-data/example_human_Illumina.pe_1.fastq --reads test-data/example_human_Illumina.pe_2.fastq --reference test-data/chr20.fa
Rabix requires Python 2.7 or 3.x to run.
There are several external dependencies for rabix.
The first one is Docker (v1.5 or later)
for running command line bioinformatics tools.
Second one is a JavaScript interpreter:
you can look for available options on
PyExecJS,
but the easiest way is probably to install something like PhantomJS or NodeJS
from your distro's repo.
Finally you should install libyaml development package.
If you are running recent Ubuntu (14.04 or newer), the following should setup your system:
$ sudo apt-get install python-dev python-pip docker.io phantomjs libyaml-dev
although you'll probably want to install newer version of Docker from the above link.
Now we can install rabix via pip:
$ pip install rabix
If you are using Anaconda there might be an issue with a version of requests,
so you should create separate environment with requests 2.2.1,
prior to running pip:
$ conda create -n rabix pip requests=2.2.1
$ source activate rabix
Try running rabix command to see if everything went well.
There are two executables installed with rabix package: rabix - the executor of apps and workflows, and rabix-tools which is command line suit with various utilities.
Main argument for rabix command is a URI (local file path or HTTP URL) to a JSON document that describes an app.
Rabix supports
JSON pointer
spec, so you can reference a description within larger JSON document: rabix "apps.json#my_app_3"
When you run rabix with tool description document, you'll be presented with arguments you need to fill in in order to run it so for example:
rabix https://s3.amazonaws.com/rabix/rabix-test/bwa-mem.json
will produce the following output:
Usage:
rabix <tool> [-v...] [-hcI] [-d <dir>] [-i <inp>] [--resources.mem=<int> --resources.cpu=<int>]
[-- --reads=<file>... --reference=<file> [--minimum_seed_length=<integer>] [--min_std_max_min=<array_number_separator(None)>...]...]
We see a lot of options here, but ideally, we'll be dealing with workflows that have lot of options pre-populated, so that we only need to supply few additional arguments such as yours input files or similar.
Let's try running this tool with some example files:
$ rabix -v https://s3.amazonaws.com/rabix/rabix-test/bwa-mem.json -- \
--reference https://s3.amazonaws.com/rabix/rabix-test/chr20.fa \
--reads https://s3.amazonaws.com/rabix/rabix-test/example_human_Illumina.pe_1.fastq \
--reads https://s3.amazonaws.com/rabix/rabix-test/example_human_Illumina.pe_2.fastq
Once again, we can supply either paths to local files or HTTP URLs.
Of course of we wanted to type down command line arguments for command line programs, we wouldn't need a middleman. Here is a more meaningful example:
rabix -v -i https://s3.amazonaws.com/rabix/rabix-test/inputs-workflow-remote.json \
https://s3.amazonaws.com/rabix/rabix-test/bwa-mem-workflow.json
Now we have entire workflow defined in one file and all the options we want to apply to that workflow in another, specified after -i option.
These are simple JSON files that you can write yourself, or more conveniently, use UI on rabix.org.
When running an app, rabix will look for a docker images specified in the document, pull them and run according to description. If for some reason you want to pre-fetch required images, you can use "--install" switch:
$ rabix --install https://s3.amazonaws.com/rabix/rabix-test/bwa-mem.json
As mentioned earlier, rabix is an implementation of Common Workflow Language, plus a playground for future ideas. First question is whether your intended contribution related to CWL in general or rabix in particular. The way you should decide this is whether it's related to overall functionality of tools, workflows and document formats (CWL) or some aspect of implementation, auxiliary utilities, etc. (rabix). If former, you should probably present your idea on the mailing list or submit an issue to the Common Workflow Language project.
If you are interested in contributing to rabix, feel free to submit issues and pull requests. If you are contributing code, pay attention to "devel" branch as it's the place where feature branches get merged. We will merge thing to master branch periodically in sync with releases on PyPI.