WORK IN PROGRESS
This project starts a Kubernetes cluster with:
- a multi-replica Kafka stateful set in kraft mode
- a people app simulating people needing elevators rides on various floors within a building
- a building app handling up/down elevator buttons being pressed
- an Apollo graphql server with subscriptions to kafka topics
- a NextJS dashboard that visualizes the real-time events
- all resources running in a Minikube kubernetes cluster
export DOCKER_USERNAME=<your docker username>
minikube start
docker build -t $DOCKER_USERNAME/kafql-kafka kafka
docker build -t $DOCKER_USERNAME/kafql-graphql graphql
docker build -t $DOCKER_USERNAME/kafql-building building
docker build -t $DOCKER_USERNAME/kafql-people people
docker build -t $DOCKER_USERNAME/kafql-dashboard dashboard
docker push $DOCKER_USERNAME/kafql-kafka
docker push $DOCKER_USERNAME/kafql-graphql
docker push $DOCKER_USERNAME/kafql-building
docker push $DOCKER_USERNAME/kafql-people
docker push $DOCKER_USERNAME/kafql-dashboard
sed -e "s|docker_username|$DOCKER_USERNAME|g" kafql.yml | kubectl apply -f -
kubectl config set-context --current --namespace=kafql
If you just want to stop the pods, scale down the replicas:
TODO scale down producer/dashboard/graphql
kubectl scale statefulsets kafka --replicas=0
Note that scaling down a StatefulSet can only occur when all associated replicas are running and ready. See this section of the Kubernetes docs for more details.
Scale the replicas back up once you are ready to start again:
kubectl scale statefulsets kafka --replicas=3
TODO scale up /dashboard/graphql/people/building/elevator
Here are the steps to delete all components in the cluster and in your local Docker install:
docker rmi $DOCKER_USERNAME/kafql-kafka
docker rmi $DOCKER_USERNAME/kafql-graphql
docker rmi $DOCKER_USERNAME/kafql-building
docker rmi $DOCKER_USERNAME/kafql-people
docker rmi $DOCKER_USERNAME/kafql-dashboard
kubectl delete -f kubernetes/kafql.yml
Remove the elevator image if you created it:
docker rmi $DOCKER_USERNAME/kafql-elevator
Keep in mind that the Docker images will still be available in your remote Docker Hub repo.
Use the following commands to delete the shared storage on the Minikube node:
minikube ssh
sudo rm -rf /mnt/data
This system is powered by simulators that listen to graphQL subscriptions and publish graphql mutations. GraphQL operations are connected to kafka topics. There are two required simulators: people and building (both are included). A third elevator simulator will be created by you, and this can be done in any language you like. The only requirements are that your simulator:
- is containerized and included in the kubernetes configuration file (so that it runs in the cluster)
- subscribes to the appropriate graphQL subscriptions. The app will receive events regarding elevator requests on specific floors, etc.
- sends graphQL mutations to move the elevator(s) up/down
Below are commands to create a typescript-based node app that can be used as the basis for a new elevator simulator:
mkdir elevator && cd elevator
npm init -y
npm i ts-node typescript apollo-boost graphql-tag graphql
npx tsc --init