/shopping-cart-k8s

Service Mesh patterns for Microservices

Primary LanguageC#MIT LicenseMIT

Shopping Cart Application with Microservices Approach

Building microservices application (Shopping Cart Application - Polyglot for services) using Kubernetes + Istio with its ecosystem parts.

Disclamation

  • Should have MINIKUBE_HOME environment variable in your machine, and value should point to C:\users\<your name>\
  • Should run powershell script to create minikube machine in C: drive.
  • If it threw the exception that it couldn't find out minikube machine in Hyper-V so just simply delete everything in <user>/.minikube folder, but we could keep cache folder to avoid download everything from scratch, then runs it subsequently.

Table of contents

Technical Stack

Setup Local Kubernetes

  • Using minikube for Windows in this project, but you can use Mac or Linux version as well

  • Download the appropriate package of your minikube at https://github.com/kubernetes/minikube/releases (Used v0.25.2 for this project)

  • Install it into your machine (Windows 10 in this case)

  • After installed minikube, then run

Hyper-V

> minikube start --kubernetes-version="v1.9.0" --vm-driver=hyperv --hyperv-virtual-switch="minikube_switch" --cpus=4 --memory=4096 --v=999 --alsologtostderr

Then start with full option

> minikube start --extra-config=apiserver.Features.EnableSwaggerUI=true,apiserver.Authorization.Mode=RBAC,apiserver.Admission.PluginNames=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota --v=999 --alsologtostderr

VirtualBox v5.2.8

> minikube start --vm-driver="virtualbox" --kubernetes-version="v1.10.0" --cpus=4 --memory 4096 --extra-config=apiserver.authorization-mode=RBAC,apiserver.Features.EnableSwaggerUI=true,apiserver.Admission.PluginNames=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota --v=7 --alsologtostderr

Setup Istio

> kubectl create -f install/kubernetes/istio.yaml

or

> kubectl create -f install/kubernetes/istio-auth.yaml

Notes: set istio\bin\istioctl.exe to the PATH of the windows.

Setup Ambassador

  • If you're running in a cluster with RBAC enabled:
> kubectl apply -f https://getambassador.io/yaml/ambassador/ambassador-rbac.yaml
  • Without RBAC, you can use:
> kubectl apply -f https://getambassador.io/yaml/ambassador/ambassador-no-rbac.yaml
  • If you're going to use Ambassador, then run as following script
> cd k8s
> istioctl kube-inject -f istio-shopping-cart.yaml | kubectl apply -f -
> kubectl apply -f ambassador-service.yaml

Notes: for some reason, I couldn't run the no-rbac mode on my local development.

Dashboard

> minikube dashboard

> kubectl get svc -n istio-system
> export GATEWAY_URL=$(kubectl get po -l istio-ingress -n istio-system -o jsonpath='{.items[0].status.hostIP}'):$(kubectl get svc istio-ingress -n istio-system -o jsonpath='{.spec.ports[0].nodePort}')
> curl $GETWAY_URL

Install and Work with Helm (Kubernetes Package Manager)

> choco install kubernetes-helm
> cd <git repo>
> helm init
> helm repo update
> helm version
  • Install RabbitMq
> helm install --name my-rabbitmq --set rbacEnabled=false stable/rabbitmq

Now we can use amqp://my-rabbitmq.default.svc.cluster.local:5672 on Kubernetes Cluster, but what if we want to leverage it for the local development. The solution is port-forward it to our localhost as

> kubectl get pods | grep rabbitmq | awk '{print $1;}'
> kubectl port-forward <pod name just got> 15672

Or port-forward 5672 on Kubernetes (amqp protocol) to localhost:5672

> kubectl port-forward <pod name just got> 1234:5672

Now we have

> amqp://root:letmein@127.0.0.1:1234
  • Install Redis
> helm install --name my-redis stable/redis

Build Our Own Microservices

  • Run
> minikube docker-env
  • Copy and Run
> @FOR /f "tokens=*" %i IN ('minikube docker-env') DO @%i

From now on, we can type docker images to list out all images in Kubernetes local node.

  • Build our microservices by running
> powershell -f build-all.ps1
  • Then if you want to just test it then run following commands
> cd k8s
> kubectl apply -f shopping-cart.yaml

  • In reality, we need to inject the sidecards into microservices as following
> cd k8s
> istioctl kube-inject -f shopping-cart.yaml | kubectl apply -f -
  • In Deployment

  • In each Pod

Available Microservices

  • Get host IP
> minikube ip
  • Get Ambassador port
> kubectl get svc ambassador -o jsonpath='{.spec.ports[0].nodePort}'
  • Finally, open browser with <IP>:<PORT>

  • Microservices

    • Catalog service: www.<IP>.xip.io:<PORT>/c/swagger/. For example, http://www.192.168.1.6.xip.io:32097/c/swagger/
    • Supplier service: www.<IP>.xip.io:<PORT>/s/
    • Security service: www.<IP>.xip.io:<PORT>/id/account/login or www.<IP>.xip.io:<PORT>/id/.well-known/openid-configuration
    • Email service: www.<IP>.xip.io:<PORT>/e/

Develop A New Service

  • Build the whole application using
> powershell -f build-all.ps1
  • Then run
> kubectl delete -f shopping-cart.yaml
  • And
> kubectl apply -f shopping-cart.yaml
  • Waiting a sec for Kubernetes to refresh.

Metrics Collection, Distributed Tracing, and Visualization

Setup Prometheus

> cd istio\install\kubernetes\addons\
> kubectl apply -f prometheus.yaml

Setup Grafana

> cd istio\install\kubernetes\addons\
> kubectl apply -f grafana.yaml
> kubectl -n istio-system port-forward $(kubectl -n istio-system get pod -l app=grafana -o jsonpath='{.items[0].metadata.name}') 3000:3000 &
> curl http://localhost:3000

Setup Service Graph

TODO

Setup Zipkin or Jaeger

TODO

Setup Weave Scope

  • Install and run it on the local
> kubectl apply -f "https://cloud.weave.works/k8s/scope.yaml?k8s-version=v1.9.0"
  • Then port-forward it out as following
> kubectl get -n weave pod --selector=weave-scope-component=app -o jsonpath='{.items..metadata.name}'
> kubectl port-forward -n <weave scope name> 4040
  • Go to http://localhost:4040

Tips and Tricks

  • Print out environment variables in one container
> kubectl get pods
> kubectl exec <pod name> env
  • Switch to another use-context

Let say we have a profile named minikube19, then just type the command as below

> kubectl config use-context minikube19

Switched to context "minikube19".
> minikube config set profile minikube19

Working with Ubuntu on Hyper-V

> apt-get update && apt-get install -y apt-transport-https
> curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
> cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
> apt-get update
> apt-get install -y kubectl
  • Install minikube
> curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.27.0/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
  • Alias command
> alias k='kubectl'
> alias mk='/usr/local/bin/minikube'
  • Then run following script
> minikube start --vm-driver="virtualbox" --kubernetes-version="v1.9.0" --cpus=4 --memory 4096 --extra-config=apiserver.authorization-mode=RBAC --v=7 --alsologtostderr
> sysctl net.ipv4.ip_forward=1
> iptables -t nat -A PREROUTING -p tcp -d <VM IP> --dport 30000 -j DNAT --to-destination 192.168.99.100:30000
> iptables -t nat -A POSTROUTING -j MASQUERADE
> sudo iptables -t nat -v -x -n -L
> sudo dhclient eth0 -v
> sudo apt-get install socat
> socat -v tcp-listen:30000,fork tcp:192.168.99.100:30000
  • ngrok on Ubuntu server
> wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip
> unzip ngrok-stable-linux-amd64.zip
> mk dashboard --url
> ./ngrok http 192.168.99.100:30000 -region ap
  • Open ubuntu VM port (if needed)
> sudo iptables -I INPUT -p tcp --dport 2375 -j ACCEPT
or
> sudo ufw allow 2375
then
> nc -l 2375