This repository containes experimental examples for how to customize TCE to provision clusters with additional capabilities beyond the standard offering of TCE. The repository is not well documented currently and does not follow in all cases best practices and should be used as a starting point / point of reference and not considered by any means as production ready at this point. There are 3 main folders in the Repo:
- tce-customizations - YAML / YTT templates for customizing TCE
- tce-helper-scripts - A set of scripts to help manage TCE
- carvel-packages - A set of custom packages built for use on TCE
By default TCE only creates a single Machine Deployment (MD-0) there are many use cases where a homogeneous cluster is not optimal and we need to go down a hetrogeneous cluster approach. Examples:
- Nodes of different sizes
- Nodes with Different OSes
- Nodes with GPUs and nodes without GPUs
- Nodes on different underlying networks
TCE comes with optional extensions that you can install which will provide additional capabilities to your cluster. The TCE Extensions currently are:
- Monitoring - Prometheus + Grafana
- Ingress (Layer 7 Load Balancing) - Contour
- DNS Management for services and ingress resources - External DNS
- Log Aggregation - FluentBit
- Container Registry - Harbor
- Backup and Restore - Velero
- Networking - Multus
While these extensions are available and supported, there is no OOTB way to install them automatically on every new cluster. this repo contains A way to do this
If we have purchased Tanzu Advance, we may want to automate the deployment of the different operators provided automatically on all of are clusters. Examples:
- Auto install the Tanzu RabbitMQ Operators
- Auto install the Tanzu MySQL Operator
- Auto install the Tanzu Postgres Operator
- Auto install Tanzu Build Service
Kubernetes clusters many times need to be attached to external systems. these could be Tanzu SaaS offerings or different solutions. Examples:
- Add cluster as a K8s endpoint in VROPS for monitroing
- Attach cluster to Tanzu Mission Control (TMC)
- Attach cluster to Tanzu Observability (Wavefront)
Many times we need our clusters to trust an organizations internal CA. in order to do this we must add custom YTT templates and the CA PEM file in order for it to be trusted automatically by all nodes
While TCE comes with calico and antrea, we may want a different version of one of those CNIs or an entirely different CNI. in this repo we have automations built to do:
- Cilium CNI
- Calico Operator based installation - including routable Pods and Typha
- Antrea 1.3 with the new NSX-T interworking integration