/aks-crossplane-terraform

This sample deploys and AKS cluster with Terraform and installs Crossplane to the cluster via Helm along with the Azure provider,

Primary LanguageHCLMIT LicenseMIT

page_type languages products name description urlFragment
sample
azurecli
terraform
bash
python
yaml
json
azure
azure-openai
azure-application-gateway
azure-resource-manager
azure-kubernetes-service
azure-container-registry
azure-storage
azure-blob-storage
azure-storage-accounts
azure-bastion
azure-private-link
azure-virtual-network
azure-key-vault
azure-monitor
azure-log-analytics
azure-virtual-machines
Deploy an Azure Kubernetes Service cluster with Crossplane using Terraform
This sample shows how to deploy an Azure Kubernetes Service cluster with Crossplane using Terraform
aks-crossplane-terraform

Deploy an Azure Kubernetes Service cluster with Crossplane using Terraform

In this sample, I shows how to automate the deployment via Terraform of an Azure Kubernetes Service (AKS) cluster with Crossplane and the Upbound Azure Providers.

Crossplane is an open source Kubernetes extension that transforms your Kubernetes cluster into a universal control plane. Crossplane lets you manage anything, anywhere, all through standard Kubernetes APIs. Crossplane can even let you order a pizza directly from Kubernetes. If it has an API, Crossplane can connect to it. With Crossplane, platform teams can create new abstractions and custom APIs with the full power of Kubernetes policies, namespaces, role based access controls and more. Crossplane brings all your non-Kubernetes resources under one roof. Custom APIs, created by platform teams, allow security and compliance enforcement across resources or clouds, without exposing any complexity to the developers. A single API call can create multiple resources, in multiple clouds and use Kubernetes as the control plane for everything.

For more information, see:

NOTE
You can find the architecture.vsdx file used for the diagram under the visio folder.

Prerequisites

Architecture

The following diagram shows the architecture and network topology deployed by the sample:

AKS Architecture

This project provides a set of Terraform modules to deploy thw following resources:

Test Scripts

The scripts folder contains the following files Bash scripts to test Crossplane when the deployment completed.

Composite Resource Definition

The following script creates a test CompositeResourceDefinition. Composite resource definitions (XRDs) define the schema for a custom API. The XRD spec contains all the information about the API including the group, version, kind and schema. The XRD’s name must be the combination of the plural and group. The schema uses the OpenAPIv3 specification to define the API spec. The API defines a location that must be oneOf either EU or US. Apply this XRD to create the custom API in your Kubernetes cluster.

#!/bin/bash

# Create the CompositeResourceDefinition for the VirtualMachine resource
cat <<EOF | kubectl apply -f -
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: virtualmachines.compute.example.com
spec:
  group: compute.example.com
  names:
    kind: VirtualMachine
    plural: virtualmachines
  versions:
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              location:
                type: string
                oneOf:
                  - pattern: '^EU$'
                  - pattern: '^US$'
            required:
              - location
    served: true
    referenceable: true
  claimNames:
    kind: VirtualMachineClaim
    plural: virtualmachineclaims
EOF

Adding the claimNames allows you to access this API either at the cluster level with the VirtualMachine endpoint or in a namespace with the VirtualMachineClaim endpoint. The namespace scoped API is a Crossplane Claim.

NOTE
For more details on the fields and options of Composite Resource Definitions read the XRD documentation.

You can display the installed XRD with the kubectl get xrd command.

1kubectl get xrd
2NAME                                  ESTABLISHED   OFFERED   AGE
3virtualmachines.compute.example.com   True          True      43s

You can view the new custom API endpoints running the kubectl api-resources | grep VirtualMachine command.

1kubectl api-resources | grep VirtualMachine
2virtualmachineclaims              compute.example.com/v1alpha1           true         VirtualMachineClaim
3virtualmachines                   compute.example.com/v1alpha1           false        VirtualMachine

Composition

When you access the custom API Crossplane takes their inputs and combines them with a template describing what infrastructure to deploy. Crossplane calls this template a Composition. A Composition defines all the cloud resources to deploy. Each entry in the template is a full resource definitions, defining all the resource settings and metadata like labels and annotations. This template creates an Azure LinuxVirtualMachine NetworkInterface, Subnet VirtualNetwork and ResourceGroup. Crossplane uses patches to apply your input to the resource template. This Composition takes your location input and uses it as the location used in the individual resource. Apply this Composition to your cluster.

#!/bin/bash

# Create the Composition for the VirtualMachine resource
cat <<EOF | kubectl apply -f -
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: crossplane-quickstart-vm-with-network
spec:
  resources:
    - name: quickstart-vm
      base:
        apiVersion: compute.azure.upbound.io/v1beta1
        kind: LinuxVirtualMachine
        spec:
          forProvider:
            adminUsername: adminuser
            adminSshKey:
              - publicKey: ssh-rsa
                  AAAAB3NzaC1yc2EAAAADAQABAAABAQC+wWK73dCr+jgQOAxNsHAnNNNMEMWOHYEccp6wJm2gotpr9katuF/ZAdou5AaW1C61slRkHRkpRRX9FA9CYBiitZgvCCz+3nWNN7l/Up54Zps/pHWGZLHNJZRYyAB6j5yVLMVHIHriY49d/GZTZVNB8GoJv9Gakwc/fuEZYYl4YDFiGMBP///TzlI4jhiJzjKnEvqPFki5p2ZRJqcbCiF4pJrxUQR/RXqVFQdbRLZgYfJ8xGB878RENq3yQ39d8dVOkq4edbkzwcUmwwwkYVPIoDGsYLaRHnG+To7FvMeyO7xDVQkMKzopTQV8AuKpyvpqu0a9pWOMaiCyDytO7GGN
                  example@docs.crossplane.io
                username: adminuser
            location: "Central US"
            osDisk:
              - caching: ReadWrite
                storageAccountType: Standard_LRS
            resourceGroupNameSelector:
              matchControllerRef: true
            size: Standard_B1ms
            sourceImageReference:
              - offer: debian-11
                publisher: Debian
                sku: 11-backports-gen2
                version: latest
            networkInterfaceIdsSelector:
              matchControllerRef: true
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: "spec.location"
          toFieldPath: "spec.forProvider.location"
          transforms:
            - type: map
              map: 
                EU: "Sweden Central"
                US: "Central US"
    - name: quickstart-nic
      base:
        apiVersion: network.azure.upbound.io/v1beta1
        kind: NetworkInterface
        spec:
          forProvider:
            ipConfiguration:
              - name: crossplane-quickstart-configuration
                privateIpAddressAllocation: Dynamic
                subnetIdSelector:
                  matchControllerRef: true
            location: "Central US"
            resourceGroupNameSelector:
              matchControllerRef: true
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: "spec.location"
          toFieldPath: "spec.forProvider.location"
          transforms:
            - type: map
              map: 
                EU: "Sweden Central"
                US: "Central US"            
    - name: quickstart-subnet
      base:
        apiVersion: network.azure.upbound.io/v1beta1
        kind: Subnet
        spec:
          forProvider:
            addressPrefixes:
              - 10.0.1.0/24
            virtualNetworkNameSelector:
              matchControllerRef: true
            resourceGroupNameSelector:
              matchControllerRef: true
    - name: quickstart-network
      base:
        apiVersion: network.azure.upbound.io/v1beta1
        kind: VirtualNetwork
        spec:
          forProvider:
            addressSpace:
              - 10.0.0.0/16
            location: "Central US"
            resourceGroupNameSelector:
              matchControllerRef: true
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: "spec.location"
          toFieldPath: "spec.forProvider.location"
          transforms:
            - type: map
              map: 
                EU: "Sweden Central"
                US: "Central US"
    - name: crossplane-resourcegroup
      base:
        apiVersion: azure.upbound.io/v1beta1
        kind: ResourceGroup
        spec:
          forProvider:
            location: Central US
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: "spec.location"
          toFieldPath: "spec.forProvider.location"
          transforms:
            - type: map
              map: 
                EU: "Sweden Central"
                US: "Central US"
  compositeTypeRef:
    apiVersion: compute.example.com/v1alpha1
    kind: VirtualMachine
EOF

The compositeTypeRef defines which custom APIs can use this template to create resources.

NOTE
Read the Composition documentation for more information on configuring Compositions and all the available options.

Read the Patch and Transform documentation for more information on how Crossplane uses patches to map your inputs to Composition resource templates. You can view the Composition by running the kubectl get composition command:

1kubectl get composition
2NAME                                    XR-KIND           XR-APIVERSION                     AGE
3crossplane-quickstart-vm-with-network   XVirtualMachine   custom-api.example.org/v1alpha1   77s

Composite Resources and Managed Resources

With the custom API (XRD) installed and associated to a resource template (Composition), you can access the API to create resources. Create a VirtualMachine object to create the cloud resources.

#!/bin/bash

# Variables
name="bingo"
location="EU"

# With the custom API (XRD) installed and associated to a resource template (Composition) users can access the API to create resources.
# Create a VirtualMachine object to create the cloud resources.
cat <<EOF | kubectl apply -f -
apiVersion: compute.example.com/v1alpha1
kind: VirtualMachine
metadata:
  name: $name
spec: 
  location: $location
EOF

You can the resource with the kubectl get VirtualMachine command.

NOTE
It may take up to five minutes for the resources to provision.

kubectl get VirtualMachine
NAME    SYNCED   READY   COMPOSITION                             AGE
bingo   True     True    crossplane-quickstart-vm-with-network   3m3s

This object is a Crossplane Composite Resource (XR). It’s a single object representing the collection of resources created from the Composition template. You can view the individual resources by running the kubectl get managed command.

kubectl get managed
NAME                                         READY   SYNCED   EXTERNAL-NAME   AGE
resourcegroup.azure.upbound.io/bingo-7jb4n   True    True     bingo-7jb4n     3m43s
 
NAME                                                       READY   SYNCED   EXTERNAL-NAME   AGE
linuxvirtualmachine.compute.azure.upbound.io/bingo-5h7p4   True    True     bingo-5h7p4     3m43s

NAME                                                    READY   SYNCED   EXTERNAL-NAME   AGE
networkinterface.network.azure.upbound.io/bingo-j7fpx   True    True     bingo-j7fpx     3m43s

NAME                                          READY   SYNCED   EXTERNAL-NAME   AGE
subnet.network.azure.upbound.io/bingo-b2dqt   True    True     bingo-b2dqt     3m43s

NAME                                                  READY   SYNCED   EXTERNAL-NAME   AGE
virtualnetwork.network.azure.upbound.io/bingo-pd2sw   True    True     bingo-pd2sw     3m43s

Accessing the API created all five resources defined in the template and linked them together. Look at a specific resource to see it’s created in the location used in the API.

kubectl describe linuxvirtualmachine | grep Location
    Location:                         Sweden Central
    Location:                         swedencentral

You can delete the resources with the kubectl delete VirtualMachine command.

kubectl delete VirtualMachine bingo
virtualmachine.compute.example.com "bingo" deleted

Verify Crossplane deleted the resources with kubectl get managed after a few minutes.

MIT License

Copyright (c) 2024 Paolo Salvatori

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.