Adding PLAZA genomes in Galaxy

Building a set of .yaml files that can be used as input for ephemeris to automatically add all plant genome build available in the PLAZA data warehouse and run the index-building data managers on these.

How it works

To install the data in a running instance:

Install the required data managers (located in data_managers.yaml.lock). This file is ready to be used with ephemeris:
```
   shed-tools install -t genomes/data_managers_list.yaml.lock -g $GALAXY_URL -a $API_KEY
```
Run genomes/PLAZA_get_galaxy_information.py. This script calls the PLAZA API to retrieve updated genome information and writes the output in predefined files:
- genomes.yaml file: This contains a yaml-structured file with all the information retrieved.
- genome_data_manager_run.yaml and transcriptome_data_manager_run.yaml: These files combines the genome builds information in genomes.yaml with the different data managers configuration in a way that makes these directly usable by ephemeris to install the data.
Run the data managers.This is done in 2 steps:
- Run the genome dependant data managers (genome_data_manager_run.yaml): This creates the corresponding entries in the dbkeys table, installs the genomes fasta files, annotations, and runs the index on these:
```
run-data-managers --config genome_data_manager_run.yaml -g $GALAXY_URL -a $API_KEY
```
- (OPTIONAL) If you also want to install the transcriptomes and indexes associated with these, run the transcriptome dependant data managers (transcriptome_data_manager_run.yaml). This installs the transcriptomes, linking them with the previoisly created dbkey, and finally creates the corresponding indexes based on these files:
```
run-data-managers --config transcriptome_data_manager_run.yaml -g $GALAXY_URL -a $API_KEY
```

To deploy a docker-based Galaxy instance and install the data there:

On command line
```
  cd build
  sh build.sh
```

alternatively:

  cd build
  cp build_public.sh.sample build_public.sh

  # update the content with the right url and API key

  sh build_public.sh

Requirements

python3
docker: In case of deploying a docker-based Galaxy instance.

ephemeris, which can be installed in different ways.

Install directly from repo src code. This has the advantage of installing the latest version, which is much faster to deploy a large number of genomes:
```
   pip install -e git+https://github.com/galaxyproject/ephemeris.git@dm#egg=ephemeris
```

Install with conda (use Miniconda3 in order to prevent conflicts with python3 ):

  conda create --channel bioconda --name ephemeris ephemeris 
  source ~/miniconda2/bin/activate ephemeris

Install inside a python virtualenv:

  virtualenv .venv
  . .venv/bin/activate
  pip install -U pip
  pip install -e git+https://github.com/galaxyproject/ephemeris.git@dm#egg=ephemeris

The run-data-managers methods in ephemeris requires the "watch_tool_data_dir” setting in galaxy.ini to be True (see https://ephemeris.readthedocs.io/en/latest/commands/run-data-managers.html) when running multiple data managers that are interdependent, as is the case with genome_data_manager_run.yaml and transcriptome_data_manager_run.yaml.

Adding new data managers

New data managers can be added to genomes/data_managers_genome_based.yaml or genomes/data_managers_transcriptome_based.yaml, after running genomes/PLAZA_get_galaxy_information.py these will be added in the corresponding conf file to be used with ephemeris.

how to find the ID: run it in Galaxy and check the info
to make it work you need the version at the end of the ID!
for the right parameters go into the wrapper in the repo of the tool (link in ToolShed)

Known issues

Docker instance gives error in bowtie indexing (file not found), public server did work with same script. Updating the docker image might be required