Learn Circle CI (Continuous Integration)

A step-by-step tutorial for using Circle CI for Continuous Integration & Deployment!

Note: This guide/tutorial is a vendor-specific set of instructions for how to enable CI/CD for a Node.js project hosted on GitHub. For a more general introduction to Continuous Integration, please read/follow our "Learn Travis" tutorial first: https://github.com/dwyl/learn-travis

Why?

In a nutshell: the reason you would use Circle-CI is because it's "free"¹. see: https://circleci.com/pricing

If you have a product owner / client who wants/needs their code to remain private and does not want to pay $49/month for one of the other CI/CD services, then Circle-CI is your go-to choice.

¹Most of our client private projects that use Circle-CI don't pay a penny. However one of our clients is paying for Circle-CI because tests in their 3 (actively developed) apps take up more than 1500 "free minutes" to build/test per month. So we feel they are "breaking even" on us...

What?

Run your (application/unit/integration/UI) tests using a container on Circle-CI and deploy your app to your choice of environment if they pass.

Who?

This tutorial is aimed at developers who need a way of automatically testing and deploying their private apps/code.

Prerequisites?

Basic knowledge of CI concepts. If you are new to (or "rusty") on CI, please read/follow our "Learn Travis" tutorial first: https://github.com/dwyl/learn-travis
Basic JavaScript/Node.js knowledge.
10 Minutes of Time.

No other knowledge/skill/experience is assumed or implied.

Stuck? If you have any questions related to getting started with Circle-CI or you are "stuck" deploying using the script(s) we demo here, please open an issue/question and we will try our best to help! https://github.com/dwyl/learn-circleci/issues

How

If you do not already have an App that you want to test and deploy, see:

1. Register

2. Add Project

Click the "add project" + button from the left-hand side menu. Then select the project you want Circle-CI to test for you, and click the "Setup Project" button:

You should see something like this now:

3. Setup `.circleci/config.yml` File

Next, follow the on-screen instructions to create a config.yml folder and setup your .circleci/config.yml file.

mkdir .circleci
vi .circleci/config.yml

Click on the "Copy To Clipboard" button to get the code. paste the sample code into the .circleci/config.yml file you have open in your text editor.

e.g:

Save the file, commit it and push to GitHub. e.g: https://github.com/nelsonic/circleci-hello-world-nodejs/commit/ad368321f3fa05686c97a8147f1e5570870bb527

Note: Circle-CI installs Yarn by default. This is NOT required anymore in 2018. It adds a needless step to the build process and sends your package.json to Facebook HQ. I've left it as the default for this example, but we don't use Yarn for any of our Node.js projects, there is zero benefit.

4. Start Building

Once you have created the .circleci/config.yml, scroll down the page and click on "Start building":

You should be redirected to a build progress/status page:

e.g: https://circleci.com/gh/nelsonic/circleci-hello-world-nodejs/1

The build and running the (single unit) test, took 3 seconds and ended in "SUCCESS". (it's a "hello world" example, it should be fast and pass!)

5. Status Badge!

At this point we can add a "Badge" to our project README.md

e.g: https://circleci.com/gh/nelsonic/circleci-hello-world-nodejs/edit#badges

For why this is a good idea, see: https://github.com/dwyl/repo-badges

Part 2 - Continuous Deployment

Continuous Deployment (CD) is the perfect complement to Continuous Integration (CI) and people usually use the same "pipeline" (system) to automate it. It just means that your application is automatically deployed by the CI system (in this case CircleCI) when the PR is merged.

If you are new to Continuous Deployment (CD) in general we wrote an introductory post in: https://github.com/dwyl/learn-devops

¹As always, we recommend using Heroku for "MVP" App(s) because it's much easier/faster to get started and traction. see: https://github.com/dwyl/learn-heroku and: https://circleci.com/docs/2.0/deployment-integrations/#heroku once you need "more power" than Heroku (or need to reduce costs), take a look at: https://github.com/dwyl/learn-devops

6. Pre-Deployment Checks

Ensure that you have a "start" script in your package.json. e.g:

{
  "scripts": {
    "start": "node server.js",
    "test": "node hello.test.js"
  }
}

Ensure that your app runs on your localhost before attempting to deploy it to your chosen "cloud" provider.

Run the start command:

npm start

Visit: http://localhost:5000

Ok, so it's working on localhost; we have a good "baseline". Onwards!

Note: there are several ways of starting node.js servers, (e.g: PM2, Forever or SystemD) we are using "node server.js" for simplicity. But if you are running your own infrastructure, you will have a well defined system for running/monitoring node.js.

7. Deploy!

For this section we are doing an "advanced" deployment to a DigitalOcean Instance using Dokku (which is an Open Source Heroku "clone"). It's only "advanced" in that there are a few more "steps" than deploying to Heroku, but don't worry, we have documented everything "step-by-step" so it should only take you about 10 minutes to setup.

Note: as always, if you get "stuck", just open an issue and we will try our best to help! https://github.com/dwyl/learn-circleci/issues

7.1 Add RSA Key

In order to deploy the app via SSH, we need to add the SSH (RSA) key to CircleCI. Thankfully this is a lot easier thank on Travis-CI!

In your app's Settings page,

Scroll down to the "Permissions" section.
Click on SSH Permissions
Click on the Add SSH Key button.

A "modal" window will appear which allows you to paste your Private Key.

Note: we do NOT recommend using your personal private key for deployments! rather you should create a key that is specific to the server you are deploying to and is only "known" (used) by your CI/CD system. see: https://github.com/dwyl/learn-devops

We have successfully added our SSH Deployment RSA Key!

7.2 Add Deployment Scripts to your Project

Create a /bin directory in your project.

mkdir bin

Copy the deployement scripts from: https://github.com/nelsonic/circleci-hello-world-nodejs/tree/master/bin

You can either do this manually by downloading each file to your project, or ... using a script!

7.3 Add Required Environment Variables

Add any Environment Variables you need to your .circleci/config.yml file. The IP Address of the server is required as is the DOKKU_APP name.

e.g:

environment:
   SERVER_IP_ADDRESS: 138.68.163.126
   DOKKU_APP: circlecidemo
   TRAVIS: "false"

working example: https://github.com/nelsonic/circleci-hello-world-nodejs/blob/ecfab4a49141da87f36519e50ecda593f01aaf48/.circleci/config.yml#L8

Note: if you need your environment variables to not be in Version Control, add them via the CircleCI Web Interface. more details: https://circleci.com/docs/2.0/env-vars

If you are new to Environment Variables, checkout: https://github.com/dwyl/learn-environment-variables

7.4 Add Command to Run Deployment Script

Add the following lines to your .circleci/config.yml file:

# Use BASH instead of DASH! see: https://ubuntuforums.org/showthread.php?t=1932504
- run: ls -al /bin/sh && sudo rm /bin/sh && sudo ln -s /bin/bash /bin/sh && ls -al /bin/sh
# test shell script:
- run: echo ${SERVER_IP_ADDRESS}
- run: echo -e "Host $SERVER_IP_ADDRESS\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config
- run: sh bin/deploy.sh

Working example: https://github.com/nelsonic/circleci-hello-world-nodejs/blob/ecfab4a49141da87f36519e50ecda593f01aaf48/.circleci/config.yml#L43-L48

Push your code to GitHub and let CircleCI do the rest! It usually takes a minute to deploy (depending on your dependencies)

In the build log you should see something like:

e.g: https://circleci.com/gh/nelsonic/circleci-hello-world-nodejs/18

8. Check it Worked!

In our case our DOKKU_APP was defined as circlecidemo (see: section 7.3 above) So the deployed app is available on: https://circlecidemo.ademo.app If we visit this URL in our browser we see:

Notice the "Git Hash" value: dc933373388911d9405fb452c9753f0250766bb3

This corresponds to the latest commit on the master branch: https://github.com/nelsonic/circleci-hello-world-nodejs/commits/master (GitHub only shows the first 7 characters of the has in the UI in this case: dc93337)

So it worked! (the latest master has been deployed by CircleCI!)

`done`

Relevant / Background Reading

How to run Bash Scripts on CircleCI: https://circleci.com/docs/2.0/using-shell-scripts
Permission denied running bash script ... https://stackoverflow.com/questions/33942926/circleci-permission-denied-running-bash-script

Official "Getting Started" Video?

The official get started on CircleCI 2.0 video: https://youtu.be/KhjwnTD4oec uses a Python example: https://github.com/bellkev/circleci-demo-python-django it's not really "hello world" (beginner friendly) ... So just focus on the UI/navigation, then come back here for a simple example!

dwyl/learn-circleci