ncats-kg-release-pipeline

This is the repository for driving a Jenkins Multibranch pipeline.

Each branch will have its own version of Jenkinsfile that drives the pipeline.

Available branches:

master - contains a Jenkinsfile which serves as a template for a Jenkins build. The master branch should never generate any artifacts.
test - contains a Jenkinsfile which downloads test datasets, parses them using KGX to build a KG. The output is a KG as a csv.

How do I run Jenkins

You can set up Jenkins locally, as a Docker container or run on a server.

Run Jenkins as a Docker container

To run Jenkins as a Docker container,

docker run -it \
  -p 3000:8080 \
  deepakunni3/ncats-jenkins

deepakunni3/ncats-jenkins is a Docker image that prepares an environment equipped to run KGX, and other tools, alongside Jenkins. This image is generated by this Dockerfile.

Run Jenkins as a Docker container which can spin up additional Docker containers

Use the following command if your pipeline is going to generate additional Docker containers,

docker run -it \
  -p 3000:8080 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  deepakunni3/ncats-jenkins

Run Jenkins as a Docker container and preserve generated artifacts

If your pipeline generates artifacts and you would like to preserve these then you can mount a folder from the host to $JENKINS_HOME, in this case /var/jenkins_home, such that any archived files (after a successful Jenkins build) are written to the mounted folder instead.

Ex:

 docker run -it \
  -p 3333:8080  \
  -v /my/host/folder:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  deepakunni3/ncats-jenkins

Getting started

You can use make start to run the docker container. Notice that doing so for the first time will create a new directory jenkins_home in your present working directory. This is mounted to /var/jenkins_home in the Docker container. Navigate to Jenkins in your browser and you will see a page titled "Unlock Jenkins". Use make logs to see the output of the Jenkins container, and you will find a portion of text looking like this:

Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:

557d82bb01cb4279bcf4eb4089fb312c

This may also be found at: /var/jenkins_home/secrets/initialAdminPassword

Copy and paste that password, here it is 557d82bb01cb4279bcf4eb4089fb312c, and paste it into the webpage and press Continue.

Then select the "select plugins to install" option. Unselect the "Folders" option, and then scroll down to find the "GitHub" option and select it. Then click Install. Then continue through the set up, choosing your username and password and so on. Eventually you will arrive at a page welcoming you to Jenkins and prompting you to create new jobs.

But first let's configure Jenkins to allow other users to create accounts. On the side click "Manage Jenkins" and then "Configure Global Security". Select "Allow users to sign up" and save.

Setting up a new Jenkins Pipeline

First, go to the Credentials page and create new credentials with the kind username with password, set the username to your GitHub username and the password to your GitHub password. Now create a new item. Give it a name and select "Multibranch Pipeline". Then, in the "Banch Source" section select "Add Sorce" and choose GitHub. Enter NCATS-Tangerine as the owner and ncats-kg-release-pipeline as the repository, and select the credentials you previously created. Then press Save and you're done.