This repository contains the WormBase Web application.
The repository for WormBase Mobile can be found at WormBase/website-mobile
The technical stack of WormBase website consits of:
- A web server based on a MVC framework, Catalyst and Template Toolkit
- React for some portions of the UI
- Database engines: ACeDB, MySQL, Datomic
- Nginx
- BLAST, and other bioinformatics tools
- A RESTful data API
- A RESTful search API
Italic indicates services whose deployment are managed separately from what is in this repository.
For devOps, we use Docker, docker-compose, Jenkins, AWS Elastic Beanstalk.
Development environment can be setup easily, using a shared development machine and docker-compose
.
For Legacy instructions that set up without docker or docker-compose, please visit the Manual Setup Guide.
Prerequisite:
-
Obtain access and login to the shared development instance, where data and legacy software are stored.
-
Ensure environment variable
CATALYST_PORT
andWEBPACK_SERVER_PORT
are set. -
Ensure
/usr/local/bin/
is on your $PATH, as dependencies such asdocker-compose
andyarn
are installed there.
To start your development stack:
make dev
and wait for website/Catalyst and webpack(DevServer) to start.
To shutdown your development stack cleanly:
make dev-down
make dev
appears stuck
The first time that make dev
runs, it takes longer due to installation of dependencies. Subsequent startup should take a few seconds.
The stdout is jumbled
The stdout
of docker-compose combines the stdouts of the containers. To make it easier to read, stdout of individual containers can be accessed via SERVICE=[name_of_service] make console
, where the name of service could be website, webpack, etc as found in docker-compose.yml and docker-compose.dev.yml.
docker-compose
cli commands not taking effect
The Makefile exports user-specific environment variable COMPOSE_PROJECT_NAME
to allow multiple instances of the development stack to run on the same machine. If you use docker-compose cli directly, please set COMPOSE_PROJECT_NAME
accordingly to interact with your particular development instance.
Unable to connect to ACeDB
ACeDB container isn't started as part the development stack to reduce memory footprint. Instead, we rely on a shared acedb container, which is expected to run on the docker networked wb-network
. If the shared acedb container is down, instructions to start the shared acedb container is found here.
Prettier git pre-commit hook doesn't trigger
JavaScript dependencies (such as prettier
and husky
) need to be installed on the host for code formatting and other git pre-commit hooks to work.
Compilation failure: You must configure a default store type unless you use exactly one store plugin.
This problem seems to show up occasionally, when I modify the wormbase_local.conf while the server is running. Try make dev-down
and then make dev
, and repeat a few times until the problem resolves itself.
If a local development environment is preferred, this option would allow you to setup a barebone development environment, without requiring credentials, such as those for accessing cloud resources. As a result, some features of the website will not work in this environment. But it should have enough functionalities for UI development and testing.
Prerequisite:
-
Ensure Docker (18.06.0+) and docker-compose are installed.
-
Ensure environment variable
CATALYST_PORT
andWEBPACK_SERVER_PORT
are set. -
(Updated on March 10, 2020): Copy the file wormbase_local.conf.template into a new file named
wormbase_local.conf
.
To start your development stack:
make local
and wait for website/Catalyst and webpack(DevServer) to start. (Running this for the first time will take some time, due to the download of container images and npm dependencies. Subsequent runs will be much faster.)
Please browse to the http://localhost:CATALYST_PORT
(instead of the WEBPACK_SERVER_PORT) for testing of the website.
To shutdown your development stack cleanly:
make local-down
WormBase staging site is hosted on the shared development instance. Its deployment is automated, triggered by committing to the staging branch on Github.
Continuous integration for staging environment is handled by Jenkins. Jenkins runs the jenkins-ci-deploy.sh script to for deployment and testing. For detailed setup, please visit the Jenkins web console.
WormBase production site is hosted with AWS Elastic Beanstalk. For details about customizing the production deployment, please visit the WormBase Beanstalk Guide for Website.
-
Change the WS release number in wormbase.conf, in particular,
wormbase_release
,rest_server
, andsearch_server
properties -
Update the volume snapshot for the WS release here
-
Create the release branch, such as
release/273
-
At the release branch:
make release # to build the assets (ie containers) required for deployment
make eb-create # to create and deploy to the pre-production environment
-
Login to the AWS Management Console, and navigate to Elastic Beanstalk, and then to the
website
Application.- Wait for the deployment to complete, and verify the pre-production environment is working
- If ACeDB TreeView isn't working, which seems to be caused by a race condition between setting up the file system and starting ACeDB container, the problem can be fixed by re-deploying to the same environment
make production-deploy
.
- If ACeDB TreeView isn't working, which seems to be caused by a race condition between setting up the file system and starting ACeDB container, the problem can be fixed by re-deploying to the same environment
- Wait for the deployment to complete, and verify the pre-production environment is working
- Login to the AWS Management Console, and navigate to Elastic Beanstalk, and then to the
website
Application.- Swap the URL between the pre-production environment and the current production environment
- After making sure the new production environment is working, wait until the DNS TTL passes on Nginx before shutting down the previous production environment
-
Grant permission to the instance by attaching the
wb-catalyst-beanstalk-ec2-role
. -
Prior to applying the hotfix, ensure you are at the appropriate git branch for production, such as
release/273
. -
Then run the following commands,
VERSION=[GIT_TAG_TO_BE_CREATED] make release # the tag should look like WS273.1 make production-deploy
For instances not managed by Beanstalk, deployment can be performed with:
-
Checkout the latest of the release branch, such as
release/273
-
Ensure port 5000 is available, and run:
make production-deploy-no-eb
Our development workflow can be found here:
http://wiki.wormbase.org/index.php/Development_workflow_-_webdev
Todd Harris (todd@wormbase.org)
Thanks to BrowserStack for allowing us to perform interactive cross browser and cross OS testing.