/zenbu-docker

Primary LanguageDockerfile

ZENBU

ZENBU is a data integration, data analysis, and visualization system enhanced for next-generation-sequence-tag (NGS) based data.

ZENBU allows for novel data exploration through "on-demand" data processing and interactive linked-visualizations and is able to make many-views from the same primary sequence alignment data which users can uploaded from BAM, BED, GFF and tab-text files.

Data integration, data processing, and visualization system based around three main web interfaces:

  • an expression data enhanced genome browser interface
  • a secured user system for data upload and secured data sharing
  • a data explorer interface to find and manipulate data across the many supported experimental data types and to find shared user configurations"

RIKEN ZENBU genome browser

Don't wanna bother installing a local version.
Simply visit the original ZENBU genome browser and create an user account : http://fantom.gsc.riken.jp/zenbu/

Adopt a ZENBU - docker version

docker-compose deployed ZENBU genome browser application running across multiple containers:

  • zenbu_db (mariadb-based database)
  • zenbu_www (apache-based httpd)
  • zenbu_agent (for automated cache building and uploaded data preparation)

Two images will be created : a mariadb image and a zenbu_app image. The zenbu_app image is the basis for zenbu_www_1 and zenbu_agent_1 containers' agents, webservices and running ad-hoc command line tools.

ZENBU offers a number of command line tools to manipulate the data. After successful deployement, run docker run --rm -it zenbu_app for a list of these command line tools.

Initial setup

.env

You will likely need to minimally provide several environment variables for certain functionalities (emails sent by zenbu upon account creation/modification, mounting an additional NFS-based datalake (readonly) volume, ...).
These can be provided at buildtime (-e KEY=VALUE) or via a typical docker-compose .env file

  • ZENBU source-code version : ZENBU_VERSION. Default as 2.11.2.
  • http server's name & alias: DOMAIN. Default as localhost.
  • SMTP environment variables: SMTP_HOST, SMTP_USER. Note : SMTP_PASSWORD is read from a file in /secrets/smtp_password.txt.
  • http server's admin contact and emails sent by zenbu upon account creation/modification : EMAIL. Default to admin@localhost.

DOMAIN and SMTP environment variables' values can be redefined at runtime by editing /etc/zenbu/zenbu.conf.

In addtion, when adding an NFS-based datalake (readonly) volume/mountpoint where bam, cram, bed, ... data files are readily available, you must specify :

  • the NFS server IP adress : NFS_DATALAKE_IP
  • the NFS server device (path) to be mounted : NFS_DATALAKE_DEVICE

Other (redefinable) enviroment variables at buildtime and/or runtime include:

  • MARIADB_PASSWORD_FILE & MARIADB_USER
  • ZENBU_AGENT_BUILDTIME & ZENBU_AGENT_SLEEPTIME
  • ZENBU_SRC_URL & ZENBU_SRC_PATCH
  • ZENBU_CURATOR_EMAIL, ZENBU_SESSID, ZENBU_SESSNAME, ZENBU_CACHE_DIR, ZENBU_SRC_DIR & ZENBU_USERS_DIR

See https://github.com/jessica-severin/ZENBU/tree/master/build_support and https://github.com/jessica-severin/ZENBU/blob/master/README_AUTOBUILD for details

secrets

Passwords (for smtp server and mariadb) are stored in secret files mounted at build time

  • /secrets/smtp_password.txt containing the SMTP_PASSWORD associated with SMTP_USER. No Default provided
  • /secret/db_zenbu_password.txt contains the MARIADB_PASSWORD associated with the MARIADB_USER zenbu-admin user. Default to zenbu_admin

docker-compose.yml & docker-compose.with_datalake-readonly-nfsvolume.yml

  • Basic Named volumes : volumes where mariadb, user, and cache data are stored can be customized to correspond to desired specific host's mountpoints
  • NFS-based datalake (readonly) volume : you may already have bam, cram, bed, ... data files stored in an NFS-server that you wish zenbu to have access to.
    in which case deploy using both docker-compose.yml and docker-compose.with_datalake-readonly-nfsvolume.yml files and provide its IP and device (path) as an envvar

Deployement

docker-compose --file=$PWD/docker-compose.yml --project-name zenbu up
or docker-compose --file=$PWD/docker-compose.yml --file=$PWD/docker-compose.with_datalake-readonly-nfsvolume.yml --project-name zenbu up to add an NFS-based datalake (readonly) volume

Web site

default to http://localhost/zenbu or http://DOMAIN/zenbu if the DOMAIN envvar was provided

logs

  • mariadb logs : docker logs zenbu_db_1
  • apache logs : docker logs zenbu_www_1
  • agent logs : docker logs zenbu_agent_1

Using command line tools

example : gettig your list of collabaroations via zenbu_upload

  • login/create a user account from zenbu website (e.g. http://localhost/zenbu/user/)
  • create a file called ~/.zenbu/id_hmac containing your user account email and the hmac key (obtained from http://localhost/zenbu/user) as a tab delimited key/value pair
  • run the zenbu_app image as a temprary container to which your ~/.zenbu/id_hmac credentials and zenbu_agent volumes are mounted :
    docker run --rm --volumes-from zenbu_agent_1 --volume ~/.zenbu/id_hmac:/root/.zenbu/id_hmac -it zenbu_app zenbu_upload -url http://localhost/zenbu -collabs