A minimal polyglot Microservices based calculator application built for deployment over Kubernetes.
- Philosophy
- Application
- Architecture
- Services
- Kubernetes Components
- Build and Deploy
- Important Kubernetes Commands
- Contributor
- License
I started learning Kubernetes a few months ago after wrapping up my head around Docker. According to Google, "Kubernetes is a portable, extensible open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation".
Honestly, the only simple thing I could get my hands on for first few weeks was the definition of it. No matter how hard I tried to glue things together to understand this exciting technology, I kept on failing and for a very simple reason that I was trying to too hard by looking at different articles and blog posts and was trying to accomplish all at once, however I wasn't building something around it which is unarguably the best approach to learn a new tech, yes, do it!
Its only then, that I embarked on a quest to make sure not only that I learn the concepts of Kubernetes myself, I do it in a way that makes it easier for my colleagues to understand and learn it faster than I did. The current starter-kit takes the approach of 'read-it' first along with 'do-it' next and encapsulates another hot topic 'micro services' inside the kit that will help a lot of techies to deploy and scale their micro services based deployment using docker containers and kubernetes.
The application built as a part of this quest is a simple browser based calculator whose front end is written using Vue.js and the operations (addition, multiplication etc) are served by following micro services written in various programming languages:
- Expressed service (Express.js)
- Happy service (Hapi.js)
- Bootstorage service (Spring Boot)
- Vuecalc service (Vue.js)
The Expressed and Happy services are rest api based implementations not doing much tangibly but only serve basic calculator functions as APIs. The Bootstorage micro service acts as a backend service that stores and retrieves data provided to it via the former two.
The Vuecalc service is a frontend application designed in Vue.js. It resembles a calculator, two of whose primary operations i.e. Add and Subtract are served by the expressed service and the other two operations, i.e. multiply and division are served by the happy service.
Along with the micro-services, following components are deployed in the cluster as well:
Ambassador(provided by datawire.io) is an open source Kubernetes-Native API Gateway built on the Envoy Proxy which is a native kubernetes api gateway. Redis acted as a data store for this implementation, storing the operations done on the calculator and serving them on on demand.
The complete architecture of this implementation can be summarised as:
As we can see, Ambassador is acting as a single point of entry in the Kubernetes cluster, channeling all the incoming requests to appropriate services. The various microservices are deployed at the following addresses inside the cluster:
| Service | Cluster Address | Cluster Port | API endpoint |
|---|---|---|---|
| Vuecalc | vuecalc-svc | 2000 | / |
| Expressed | express-svc | 3000 | /api/express/ |
| Happy | happy-svc | 4000 | /api/happy/ |
| Bootstorage | bootstorage-svc | 5000 | /api/bootstorage/ |
Each one of the microservice has liveness and readiness probes inbuilt for the cluster to determine their health and update the users about any potential downtime. The probes are configured for each service at the following endpoints:
| Service | Liveness Probe | Readiness Probe |
|---|---|---|
| Vuecalc | / | / |
| Expressed | /api/express/healthz | /api/express/healthz |
| Happy | /api/happy/healthz | /api/happy/healthz |
| Bootstorage | /api/bootstorage/healthz | /api/bootstorage/healthz |
These probes are configured in the individual deployment configurations as follows:
containers:
- image: expressed:latest
imagePullPolicy: Never
name: express-svc
env:
- name: MY_SECRET
valueFrom:
secretKeyRef:
name: expressed-env
key: secret
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: expressed-config
key: log_level
ports:
- containerPort: 3000
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /api/express/healthz
port: 3000
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
livenessProbe:
failureThreshold: 3
httpGet:
path: /api/express/healthz
port: 3000
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1 All the services are currently deployed under the default namespace to keep things simple. However we strongly suggest that users should opt for different namespaces specially if they are planning to deploy their DEV, QA and UAT environments all on the same cluster (of course assuming PROD is going to be an independent cluster).
The different services created for this starter-kit are summarised in the following diagram:
This is a vue-cli 3.0 based Vue.js application which renders the calculator UI as shown below:
The service is composed of 'home', 'calculator' and 'history' views. It uses Axios library to do all the API calls to the kubernetes cluster. It uses a sound architecture with separate components and services for different aspects of the interface. The 'history' view shows a list of recent operations done over the calculator application by fetching them from the 'Bootstorage' service located inside the cluster. The commands to build and run the service independently are specified in the concerned readme.md file in the service's codebase.
It is an Express.js (v4.16.x) based web application serving basic APIs for addition and subtraction. It is a containerised service and deployed inside the cluster as a pod eligible to horizontally scale under extreme conditions. The Vuecalc service calls the apis from this service when it has to do an add or subtract operation, once the operation is complete an async call is placed to the Bootstorage service to store the last performed operation in the Redis data store. Build and run commands are shared in readme.md.
It is a Hapi.js (v17.8.x) based application having two basic APIs for multiplication and division. It is also a containerised service and similar to the Expressed service, is deployed internally in the kubernetes cluster for serving the apis. It also places an async call to the 'Bootstorage' service to store the last executed operation on Redis. Build and run commands are shared in readme.md.
It is a Java (v1.8) and Spring Boot (v2.x) based microservice that has two primary purposes:
- Store executed operations inside Redis, and
- Return the list of operations stored in Redis sorted in the order of execution (last operation at top).
Redis has been used in the cluster as a statefulset which has persistence enabled and uses persistent volume claim templates in the cluster to store the Redis dump files which allows the cluster to make sure Redis always retains the values in between cluster outages too. The Redis connection parameters are dynamically passed to the pod from the ENV variables which are provided to it using the config maps. Build and run commands are shared in readme.md.
Following kubernetes components were used during the development of this starter kit:
- Services
- Deployments
- Pods
- StatefulSets
- Cronjob
- Secrets
- Config-Maps
- Persistent Volumes
- Persistent Volumes Claims
- Storage Class
- Kubernetes Dashboard
We are assuming that there is an existing kubernetes cluster already running on one of the renowned cloud hosting service provider -
- Google cloud (Kubernetes Engine)
- MS Azure (Kubernetes Cluster Service)
- AWS (AKS)
Or, if this is not the case, you can try Docker-for-desktop, it allows us to create a sample kubernetes cluster with a single node inside it. Even though it only has a single node out of the box, it suffices the complete development and testing effort needed to run an application on kubernetes cluster before we actually go to the cloud. We suggest you try your configurations first locally and once it works as expected then move to the cloud to save unnecessary hosting cost.
Some of the important components used for current project's development are explained in sections below.
ConfigMaps allow us to decouple configuration artifacts from image content to keep containerized applications portable. For the purpose of demonstration, the expressed and happy service both use a config map of their own to pass ENV var log_level to them. The yaml look as follows:
expressed-cmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: expressed-config
namespace: default
data:
log_level: INFOhappy-cmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: happy-config
namespace: default
data:
log_level: DEBUGThese configurations are referred inside the deployment specifications as follows:
expressed.yaml
template:
metadata:
labels:
run: express-svc
spec:
containers:
- image: expressed:latest
imagePullPolicy: Never
name: express-svc
env:
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: expressed-config
key: log_levelLike a Deployment, a StatefulSet manages pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable as each has a persistent identifier that it maintains across any rescheduling.
A StatefulSet operates under the same pattern as any other Controller. We define our desired state in a StatefulSet object, and the StatefulSet controller makes any necessary updates to get there from the current state.
StatefulSets are valuable for applications that require one or more of unique network identifiers or persistent storage or ordered, graceful deployment and scaling. In our case we had the need to store the data for various operations happening in the calculator application so that we can show them when user goes to the history section of the application. Here is how the configuration yaml for it looks like:
redis.yaml
---
apiVersion: v1
kind: Service
metadata:
labels:
run: redis
name: redis
namespace: default
spec:
ports:
- port: 6379
name: redis
clusterIP: None
selector:
app: redis
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
labels:
run: redis
name: redis
namespace: default
spec:
selector:
matchLabels:
app: redis
serviceName: redis
replicas: 1
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:3.2-alpine
imagePullPolicy: IfNotPresent
args: ["--requirepass", "calculator", "--appendonly", "yes", "--save", "900", "1", "--save", "30", "2"]
ports:
- containerPort: 6379
name: redis
volumeMounts:
- name: redis-volume
mountPath: /data
volumeClaimTemplates:
- metadata:
name: redis-volume
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 50MiThe Redis runs on default port 6379 however its access is password protected and needs the password calculator for any access within the cluster. The appendonly command instructs Redis to create a dump file and append to it every 900 seconds if single edit has been made or every 30 seconds if 2 edits have been made.
The volume that gets mounted points us to a volume claim which is being dynamically allocated to us using the persistent volume claim template (pvc template) definition. The pvc-template instructs the cluster to create a storage on disk and allocate the same to Redis each time it reboots. This way we do maintain the state of Redis deployment which is crucial since it is acting like the data store for our use case. You can easily replace it with MySql or any other database of your choice, but please bear in mind that you will need to do certain modifications to the Bootstorage service code in order to support the new storage mechanism.
A Cron Job creates Jobs on a time-based schedule. One CronJob object is like one line of a crontab (cron table) file. It runs a job periodically on a given schedule, written in Cron format.
In our case, we have written a cronjob that invokes the following API every 15 minutes:
http://bootstorage-svc:5000/api/bootstorage/deletelru
The purpose of this api (written in the Bootstorage service) is to delete the oldest operation that was performed on the calculator. This way we keep the footprint of our data store small and at the same time have a perfectly valid use case to show how a cronjob can be executed in the application to do a recurring operation. The yaml for the same looks as follows:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: cronjob
spec:
concurrencyPolicy: Forbid
jobTemplate:
spec:
template:
metadata:
labels:
app: cronjob
spec:
containers:
- name: cronjob
image: spotify/alpine:latest
imagePullPolicy: Always
command:
- curl
args:
- http://bootstorage-svc:5000/api/bootstorage/deletelru
restartPolicy: Never
schedule: '*/15 * * * *'
successfulJobsHistoryLimit: 3The parameter successfulJobsHistoryLimit makes sure that we see exactly 3 entries of the last successful executions of the cronjob in our kubernetes history either viewed via kubectl or on the dashboard. This cronjob also presents a classic example where a task defined in one of your micro service needs to execute periodically and you were hesitant to create an entire pod just for the sake of running the task, using this approach you can easily instruct any running pod via the API call which is executed inside a ~35MB alpine container spun up at runtime.
Ambassador is a production tested, open source API Gateway that exposes the power of Envoy Proxy in Kubernetes. Ambassador relies entirely on Kubernetes for reliability, availability, and scalability. For example, Ambassador persists all state in Kubernetes, instead of requiring a separate database. Ambassador has virtually no moving parts, and delegates all routing and resilience to Envoy Proxy and Kubernetes, respectively. It integrates the features teams need for microservices, including authentication, rate limiting, observability, routing, TLS termination, and more.
In our case, we decided to deploy Ambassador service as a LoadBalancer listening on port 80. The configuration that was responsible for the routing of incoming calls to independent micro services is as follows:
apiVersion: v1
kind: Service
metadata:
labels:
service: ambassador
name: ambassador
annotations:
getambassador.io/config: |
---
apiVersion: ambassador/v0
kind: Mapping
name: vuecalc-svc_mapping
prefix: /
service: vuecalc-svc:2000
---
apiVersion: ambassador/v0
kind: Mapping
name: express-svc_mapping
prefix: /express/
rewrite: /api/express/
service: express-svc:3000
---
apiVersion: ambassador/v0
kind: Mapping
name: happy-svc_mapping
prefix: /happy/
rewrite: /api/happy/
service: happy-svc:4000
---
apiVersion: ambassador/v0
kind: Mapping
name: bootstorage-svc_mapping
prefix: /bootstorage/
rewrite: /api/bootstorage/
service: bootstorage-svc:5000
spec:
type: LoadBalancer
ports:
- name: ambassador
port: 80
targetPort: 80
selector:
service: ambassadorFor services such as Expressed, Happy and Bootstorage we are rewriting the incoming request path to /api/* and the service mappings are following the convention of service-name:port where the ports are those allocated to these services inside the cluster. There are different ways how developers use the routing configurations, some of us prefer to put this information as a kubernetes-ambassador annotation inside the respective deployment configurations but we personally like to keep them all at a single place inside the ambassador configuration file and that's exactly what you will notice in the configs.
Dashboard is a web-based Kubernetes user interface. You can use Dashboard to deploy containerized applications to a Kubernetes cluster, troubleshoot your containerized application, and manage the cluster resources. You can use Dashboard to get an overview of applications running on your cluster, as well as for creating or modifying individual Kubernetes resources (such as Deployments, Jobs, DaemonSets, etc).
Please follow this link to learn more about how to set up the dashboard for your cluster and make use of a decent web interface to administer the kubernetes cluster. Here are a few screen shots of how the calculator application components look like on our dashboard after successful deployment:
Overview
Resources
Services
Deployments
Pods
StatefulSets
CronJobs
Config And Storage
There are some assumptions that we are making (before you can run the build and deploy command) like you have the following pre-installed on the node/environment where this cluster is being set up -
Once these are setup, please execute the following command from the root directory of this repo:
sh build-deploy.sh
This command is equipped to prepare all the docker containers needed for the setup of this application. It will create them all one by one and finally execute the configurations placed in the k8s directory which will result in the creation of various artifacts on the kubernetes cluster.
Here are some basic commands for using Kubernetes command line utility:
- kubectl proxy (Run kubernetes dashboard)
- kubectl describe secret admin-user-token-6kchd -n kube-system (Generate token)
- kubectl create -f k8s (Deploy to kubernetes)
- kubectl get deployments (Check existing deployments)
- kubectl get services (Check existing services)
- kubectl set image deployments expressed expressed=expressed:latest (Update image for Deployment)
- kubectl rollout status deployments expressed (Check status of updated rollout)
- kubectl scale --replicas=0 deployment express-svc (Scale down any deployment)
- kubectl scale --replicas=1 deployment express-svc (Scale up a deployment)
- docker image prune (Remove all dangling images)
This project is licensed under the terms of the MIT license.