Leonardo serves as a way to launch compute within the Terra security boundary. It does so via multiple different cloud hardware virtualization mechanisms, currently leveraging only the Google Cloud Platform.
Leonardo supports launching the following services for compute:
- Spark clusters through Google Dataproc
- Virtual machines through Google Compute Engine
- Kubernetes 'apps' through Google Kubernetes Engine
Currently, leonardo supports the launching of custom docker images for Jupyter and Rstudio in virtual machines and Dataproc. It also supports launching applications in Kubernetes, with a spotlight on Galaxy.
- For more information on APIs, see swagger
- For more information on custom docker images, see the terra-docker repo
- For more information on applications we support in Kubernetes, see the terra-apps repo
- For more information on Galaxy, see the Galaxy Project
It is recommended to consume these APIs and functionality via the Terra UI
We use JIRA instead of the issues page on Github. If you would like to see what we are working you can visit our active sprint or our backlog on JIRA. You will need to set-up an account to access, but it is open to the public.
Clone the repo.
$ git clone https://github.com/DataBiosphere/leonardo.git
$ cd leonardo
The instructions to run Leo locally are maintained in this confluence article. It may ask you to make an account, but no permissions are required to view.
Ensure docker is running. Spin up MySQL locally:
$ ./docker/run-mysql.sh start leonardo
Note, if you see error like
Warning: Using a password on the command line interface can be insecure.
ERROR 2003 (HY000): Can't connect to MySQL server on 'mysql' (113)
Warning: Using a password on the command line interface can be insecure.
ERROR 2003 (HY000): Can't connect to MySQL server on 'mysql' (113)
Warning: Using a password on the command line interface can be insecure.
ERROR 2003 (HY000): Can't connect to MySQL server on 'mysql' (113)
Run docker system prune -a. If the error persists, try restarting your laptop.
Build Leonardo and run all unit tests.
export JAVA_OPTS="-Dheadless=false -Duser.timezone=UTC -Xmx4g -Xss2M -Xms4G"
sbt clean compile "project http" test
You can also run a particular test suite, e.g.
sbt "testOnly *LeoAuthProviderHelperSpec"
or a particular test within a suite, e.g.
sbt "testOnly *LeoPubsubMessageSubscriberSpec -- -z "handle Azure StopRuntimeMessage and stop runtime""
where map is a substring within the test name.
If you made a change to the leonardo Db by adding a changeset xml file, and then adding that file path to the changelog
file, you have to set initWithLiquibase = true in the leonardo.conf file for these changes to be reflected in the unit
tests. Once youare done testing your changes, make sure to switch it back to initWithLiquibase = false, as this can do
some damage if you are running local Leo against Dev!
Once you're done, tear down MySQL.
./docker/run-mysql.sh stop leonardo
Do docker restart leonardo-mysql if you see java.sql.SQLNonTransientConnectionException: Too many connections error
- Running tests against FIAB Checking FIAB mysql (fina password in /etc/leonardo.conf in firecloud_leonardo-app_1 container)
docker exec -it firecloud_leonardo-mysql_1 bash
root@2f5efbd4f138:/# mysql -u leonardo -pLearn more about scalafmt
sbt scalafmtAll
To install git-secrets
brew install git-secrets
To ensure git hooks are run
cp -r hooks/ .git/hooks/
chmod 755 .git/hooks/apply-git-secrets.sh
To build jar and leonardo docker image
./docker/build.sh jar -d build
To build jar and leonardo docker image
and push to repos broadinstitute/leonardo
tagged with git hash
./docker/build.sh jar -d push