/awesome-kubernetes

Primary LanguageSmartyGNU General Public License v3.0GPL-3.0

1. Prerequisites

  • Clone this repository and pull all submodules
git clone https://github.com/Alliedium/awesome-kubernetes.git
git submodule init
git submodule update
  • Create a cluster with a registry
k3d cluster create demo --k3s-arg "--disable=traefik@server:*" --registry-create demo-registry:0.0.0.0:12345 --port 7080:8080@loadbalancer
  • Build and push a Docker image
cd ./external/springboot-api-rest-example/
docker build --file ./api/docker/Dockerfile.prod -t localhost:12345/example-api:0.1.0 ./api
docker push localhost:12345/example-api:0.1.0

2. Deployment variants

Example Details
Example 1 Single pod with sidecar, no persistence due to ephemeral volume
Example 2 Only pods, no persistence due to ephemeral volume
Example 3 Added services and persistence via PVC
Example 4 ReplicaSets, Readiness and Liveness probes
Example 5 Deployment, StatefulSet, ConfigMap, Secret

Remarks concerning secrets

For the last example above as well as for some examples below we use manifests for Kubernetes Secrets. And secret values in these manifests are hard coded. But this is just for simplicity, i.e. it is not how secrets are to be handled in real production. The reason is that these hard coded values can be seen by any person having access to respective Git repository which contradicts with an idea of secrets being a secret.

There are a few different ways to deal with the problem. Here are just two of them (without claiming that these two are the only onces that should be used):

  • Using Sealed Secrets. Here we store in Git a special kind of Kubernetes resources named SealedSecret. Its manifest contains encrypted values that can be decrypted only by the controller running in the target Kubernetes cluster and nobody else (not even the original author). This way only the controller is able to obtain the original Secret from the SealedSecret (within the cluster). Sealed Secrets use asymmetric cryptography to encrypt secrets with a public key while the private key used for decryption is only known to the controller.
  • Using AWS Secrets Manager along with Kubernetes. We can use a CSI driver, namely its AWS-based implementation called AWS Secrets and Configuration Provider deployed inside AWS EKS Kubernetes cluster (thus, this is an example of how this may be done in production in a cloud). Here we store in Git a special kind of Kubernetes resources named SecretProviderClass. Its manifest contains information about your secrets and how to display them in the Amazon EKS pod which references these secrets. And, for example, it is possible to sync secrets as a Kubernetes Secret. Another option is to mount secrets as files on the pod filesystem.

Both of these approaches allow us to avoid storing original secret values directly inside manifests. However there are cases when it is acceptable to store Secret manifests (along with the original secret values) directly inside Git repository. This is when manifests are used only for development purposes (some experiments for instance) and are designed for being deployed in a development cluster and not in production. However the main problem with such an approach is that it can lead to having two sources of truth (read two sets of manifests) - one for development and one for production. In our opinion such approach should be avoided as much as possible (instead we should use production ways to deal with secretes even for development). With all that being said, when such insecure is used all the secret values may be shared by developers and the secret values are not really hidden. To make the rest of the manifests as environment agnostic as possible we should use Kubernetes Secrets for deploying them to comply with production configurations. Pods just reference these secrets not knowing what their sources are thus making the rest of the manifests the same both for both development and production.

For more details see Good practices for Kubernetes Secrets and the blog How to use AWS Secrets & Configuration Provider with your Kubernetes Secrets Store CSI driver.

3. Backup jobs configuration variants

Both examples below assume that the Example 5 is already deployed and not cleaned up from the cluster.

Example Details
Example 6 Simple job with Minio S3 storage
Example 7 CronJob with AWS S3 storage

4. Installing useful tools in Kubernetes

Example Details
Example 8 Installing pgAdmin

5. Using aready existing Helm charts and operators, implementing new Helm charts

Example Details
Example 9 Installing PostgreSQL with metrics view via Grafana
Example 10 Installing scalable PostgreSQL via Kubernetes operator, implementing Helm chart for Spring Boot API application

6. Ingress

Example Details
Example 11 NGINX Ingress examples