- Why should we consider using Vcluster? Our requirement is to establish multiple environments for developers to facilitate development, testing, as well as regression and performance tests.
- The crucial aspect is ensuring that these environments closely mimic the structure of our staging and production environments, which are based on Kubernetes. Instead of relying on Kubernetes namespaces to create these environments, We opt to offer developers a solution that provides them with an environment that closely resembles a real Kubernetes cluster. This is where Vcluster comes into play.
- Abstract
- Table Of Contents
- ๐ vcluster overview
- ๐ Solution overview
- ๐ Bootstrap EKS cluster using CDK EKS Blueprints
- ๐ Create vcluster
- ๐ Expose vcluster using Network Laoad Balancer
- ๐ Cleanup
-
The bootstrap provisions EKS cluster with required AddOns using CDK EKS blueprints
new VpcCniAddOn(), new MetricsServerAddOn(), new KarpenterAddOn(), new AwsLoadBalancerControllerAddOn(), new EbsCsiDriverAddOn(),
-
Cluster provider
- Fargate to deploy Karpenter
- Karpenter simplifies Kubernetes infrastructure with the right nodes at the right time.
- Create two vclusters with namepsace
app1
andapp2
$ ./demo/create-vcl.sh app1 $ ./demo/create-vcl.sh app2
-
Create NLB service
โ k apply -f demo/app1/service.yaml service/app1-lb created โ k apply -f demo/app2/service.yaml service/app2-lb created โ k get svc -n app1 app1-lb NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE app1-lb LoadBalancer 172.20.150.105 k8s-app1-app1lb-bb32c11098-3381306256798df4.elb.ap-southeast-1.amazonaws.com 443:32392/TCP 30h โ k get svc -n app2 app2-lb NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE app2-lb LoadBalancer 172.20.78.127 k8s-app2-app2lb-4690ffbcfe-bfb88a1245728e8a.elb.ap-southeast-1.amazonaws.com 443:31510/TCP 49s
-
Create CName record point to the NLB DNS
โ multi-tenancy-in-eks-using-vcluster git:(master) โ ./demo/r53-record.sh create app2 โ multi-tenancy-in-eks-using-vcluster git:(master) โ ping app2-eks.simflexcloud.com PING k8s-app2-app2lb-4690ffbcfe-bfb88a1245728e8a.elb.ap-southeast-1.amazonaws.com (13.250.162.120): 56 data bytes
-
Now we can connect to the vcluster app1 and app2 using their expose endpoint
-
https://app2-eks.simflexcloud.com
โ vcluster connect app2 -n app2 --server=https://app2-eks.simflexcloud.com --update-current=false done โ Virtual cluster kube config written to: ./kubeconfig.yaml - Use `kubectl --kubeconfig ./kubeconfig.yaml get namespaces` to access the vcluster
-
Deploy
echo
andguestbook
projectโ ka2 apply -f demo/app2/vcluster ingress.networking.k8s.io/echo created deployment.apps/echo created service/echo created ingress.networking.k8s.io/guestbook created service/redis-leader created deployment.apps/redis-leader created service/redis-follower created deployment.apps/redis-follower created service/frontend created deployment.apps/frontend created
-
Get ALB DNS and point to the Web app endpoint
-
โ ka2 get ingress NAME CLASS HOSTS ADDRESS PORTS AGE echo alb * k8s-app2-dbb948e3be-939359744.ap-southeast-1.elb.amazonaws.com 80 10s guestbook alb * k8s-app2-dbb948e3be-939359744.ap-southeast-1.elb.amazonaws.com 80 10s โ ./demo/r53-record.sh create app2 k8s-app2-dbb948e3be-939359744.ap-southeast-1.elb.amazonaws.com
-
Delete vcluster
โ vcluster delete dev -n dev info Delete vcluster dev... done โ Successfully deleted virtual cluster dev in namespace dev done โ Successfully deleted virtual cluster pvc data-dev-0 in namespace dev
-
Destroy all AWS resources within this project