/terraform-aws-eks-ebs-csi-driver

A terraform module to deploy the AWS EBS CSI driver on Amazon EKS cluster.

Primary LanguageHCLApache License 2.0Apache-2.0

AWS EKS EBS CSI driver Terraform module

labyrinth labs logo

We help companies build, run, deploy and scale software and infrastructure by embracing the right technologies and principles. Check out our website at https://lablabs.io/

Terraform validate pre-commit

Description

A terraform module to deploy the AWS EBS CSI driver on Amazon EKS cluster.

Related Projects

Check out other terraform kubernetes addons.

Deployment methods

Helm

Deploy helm chart by helm (default method, set enabled = true)

Argo kubernetes

Deploy helm chart as argo application by kubernetes manifest (set enabled = true and argo_enabled = true)

Argo helm

When deploying with ArgoCD application, Kubernetes terraform provider requires access to Kubernetes cluster API during plan time. This introduces potential issue when you want to deploy the cluster with this addon at the same time, during the same Terraform run.

To overcome this issue, the module deploys the ArgoCD application object using the Helm provider, which does not require API access during plan. If you want to deploy the application using this workaround, you can set the argo_helm_enabled variable to true.

Create helm release resource and deploy it as argo application (set enabled = true, argo_enabled = true and argo_helm_enabled = true)

AWS IAM resources

To disable of creation IRSA role and IRSA policy, set irsa_role_create = false and irsa_policy_enabled = false, respectively

Storage Class

Default StorageClass is enabled by default with following definition:

{
  "name" : "ebs-csi-gp3"
  "annotations" : {
    "storageclass.kubernetes.io/is-default-class" : "true"
  }
  "allowVolumeExpansion" : true
  "volumeBindingMode" : "WaitForFirstConsumer"
  "reclaimPolicy" : "Delete"
  "parameters" : {
    "type" : "gp3"
    "encrypted" : "true"
  }
}

To override default module StorageClass behaviour use variable storage_classes. Provisioner attribute is hardcoded in EBS-CSI-driver chart and cannot be changed. For other parameters attributes please see EBS-CSI Storage Class parameters.

Potential issues after enabling StorageClass creation

With each new cluster, Amazon EKS applies a kubernetes.io/aws-ebs based StorageClass named gp2 with defined “storageclass.kubernetes.io/is-default-class” : “true” and by default module also creates StorageClass which contains same storageclass.kubernetes.io/is-default-class annotations. This may lead to errors while creating Persistent Volume Claim objects without specifying StorageClass name as described in the documentation.

There are several ways to prevent this error:

  1. delete EKS default StorageClass gp2 before PVC usage
  2. change annotation on EKS default storage class
  3. disable or modify default module StorageClass definition

Examples

See Basic example for further information.

Requirements

Name Version
terraform >= 1.0
aws >= 4.19.0
helm >= 2.6.0
kubernetes >= 2.11.0
utils >= 0.17.0

Modules

No modules.

Resources

Name Type
aws_iam_policy.this resource
aws_iam_role.this resource
aws_iam_role_policy_attachment.this resource
aws_iam_role_policy_attachment.this_additional resource
helm_release.argo_application resource
helm_release.this resource
kubernetes_manifest.this resource
aws_iam_policy_document.this data source
aws_iam_policy_document.this_irsa data source
utils_deep_merge_yaml.argo_helm_values data source
utils_deep_merge_yaml.values data source

Inputs

Name Description Type Default Required
cluster_identity_oidc_issuer The OIDC Identity issuer for the cluster string n/a yes
cluster_identity_oidc_issuer_arn The OIDC Identity issuer ARN for the cluster that can be used to associate IAM roles with a service account string n/a yes
argo_apiversion ArgoCD Appliction apiVersion string "argoproj.io/v1alpha1" no
argo_destination_server Destination server for ArgoCD Application string "https://kubernetes.default.svc" no
argo_enabled If set to true, the module will be deployed as ArgoCD application, otherwise it will be deployed as a Helm release bool false no
argo_helm_enabled If set to true, the ArgoCD Application manifest will be deployed using Kubernetes provider as a Helm release. Otherwise it'll be deployed as a Kubernetes manifest. See Readme for more info bool false no
argo_helm_values Value overrides to use when deploying argo application object with helm string "" no
argo_info ArgoCD info manifest parameter list 
[
  {
    "name": "terraform",
    "value": "true"
  }
]
 no
argo_kubernetes_manifest_computed_fields List of paths of fields to be handled as "computed". The user-configured value for the field will be overridden by any different value returned by the API after apply. list(string) 
[
  "metadata.labels",
  "metadata.annotations"
]
 no
argo_kubernetes_manifest_field_manager_force_conflicts Forcibly override any field manager conflicts when applying the kubernetes manifest resource bool false no
argo_kubernetes_manifest_field_manager_name The name of the field manager to use when applying the kubernetes manifest resource. Defaults to Terraform string "Terraform" no
argo_kubernetes_manifest_wait_fields A map of fields and a corresponding regular expression with a pattern to wait for. The provider will wait until the field matches the regular expression. Use * for any value. map(string) {} no
argo_metadata ArgoCD Application metadata configuration. Override or create additional metadata parameters map 
{
  "finalizers": [
    "resources-finalizer.argocd.argoproj.io"
  ]
}
 no
argo_namespace Namespace to deploy ArgoCD application CRD to string "argo" no
argo_project ArgoCD Application project string "default" no
argo_spec ArgoCD Application spec configuration. Override or create additional spec parameters map {} no
argo_sync_policy ArgoCD syncPolicy manifest parameter map {} no
enabled Variable indicating whether deployment is enabled bool true no
helm_atomic If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used bool false no
helm_chart_name Helm chart name to be installed string "aws-ebs-csi-driver" no
helm_chart_version Version of the Helm chart string "2.10.1" no
helm_cleanup_on_fail Allow deletion of new resources created in this helm upgrade when upgrade fails bool false no
helm_create_namespace Create the namespace if it does not yet exist bool true no
helm_dependency_update Runs helm dependency update before installing the chart bool false no
helm_description Set helm release description attribute (visible in the history) string "" no
helm_devel Use helm chart development versions, too. Equivalent to version '>0.0.0-0'. If version is set, this is ignored bool false no
helm_disable_openapi_validation If set, the installation process will not validate rendered helm templates against the Kubernetes OpenAPI Schema bool false no
helm_disable_webhooks Prevent helm chart hooks from running bool false no
helm_force_update Force helm resource update through delete/recreate if needed bool false no
helm_keyring Location of public keys used for verification. Used only if helm_package_verify is true string "~/.gnupg/pubring.gpg" no
helm_lint Run the helm chart linter during the plan bool false no
helm_package_verify Verify the package before installing it. Helm uses a provenance file to verify the integrity of the chart; this must be hosted alongside the chart bool false no
helm_postrender Value block with a path to a binary file to run after helm renders the manifest which can alter the manifest contents map(any) {} no
helm_recreate_pods Perform pods restart during helm upgrade/rollback bool false no
helm_release_max_history Maximum number of release versions stored per release number 0 no
helm_release_name Helm release name string "aws-ebs-csi-driver" no
helm_render_subchart_notes If set, render helm subchart notes along with the parent bool true no
helm_replace Re-use the given name of helm release, only if that name is a deleted release which remains in the history. This is unsafe in production bool false no
helm_repo_ca_file Helm repositories cert file string "" no
helm_repo_cert_file Helm repositories cert file string "" no
helm_repo_key_file Helm repositories cert key file string "" no
helm_repo_password Password for HTTP basic authentication against the helm repository string "" no
helm_repo_url Helm repository string "https://kubernetes-sigs.github.io/aws-ebs-csi-driver" no
helm_repo_username Username for HTTP basic authentication against the helm repository string "" no
helm_reset_values When upgrading, reset the values to the ones built into the helm chart bool false no
helm_reuse_values When upgrading, reuse the last helm release's values and merge in any overrides. If 'helm_reset_values' is specified, this is ignored bool false no
helm_set_sensitive Value block with custom sensitive values to be merged with the values yaml that won't be exposed in the plan's diff map(any) {} no
helm_skip_crds If set, no CRDs will be installed before helm release bool false no
helm_timeout Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks) number 300 no
helm_wait Will wait until all helm release resources are in a ready state before marking the release as successful. It will wait for as long as timeout bool false no
helm_wait_for_jobs If wait is enabled, will wait until all helm Jobs have been completed before marking the release as successful. It will wait for as long as timeout bool false no
irsa_additional_policies Map of the additional policies to be attached to default role. Where key is arbitrary id and value is policy arn. map(string) {} no
irsa_policy_enabled Whether to create opinionated policy to allow operations on specified zones in policy_allowed_zone_ids. bool true no
irsa_role_create Whether to create IRSA role and annotate service account bool true no
irsa_role_name_prefix The IRSA role name prefix for AWS EBS CSI controller string "ebs-csi-controller" no
irsa_tags IRSA resources tags map(string) {} no
namespace The K8s namespace in which the AWS EBS CSI driver service account has been created string "kube-system" no
service_account_create Whether to create Service Account bool true no
service_account_name The k8s EBS CSI driver service account name string "aws-ebs-csi-driver" no
settings Additional helm sets which will be passed to the Helm chart values, see https://github.com/kubernetes-sigs/aws-ebs-csi-driver/tree/master/charts/aws-ebs-csi-driver map(any) {} no
storage_classes List of the custom Storage Classes definitions list 
[
  {
    "allowVolumeExpansion": true,
    "annotations": {
      "storageclass.kubernetes.io/is-default-class": "true"
    },
    "name": "ebs-csi-gp3",
    "parameters": {
      "encrypted": "true",
      "type": "gp3"
    },
    "reclaimPolicy": "Delete",
    "volumeBindingMode": "WaitForFirstConsumer"
  }
]
 no
storage_classes_create Whether to create Storage Classes bool true no
values Additional yaml encoded values which will be passed to the Helm chart, see https://github.com/kubernetes-sigs/aws-ebs-csi-driver/tree/master/charts/aws-ebs-csi-driver string "" no

Outputs

Name Description
helm_release_application_metadata Argo application helm release attributes
helm_release_metadata Helm release attributes
iam_role_attributes EBS CSI driver IAM role atributes
kubernetes_application_attributes Argo kubernetes manifest attributes

Contributing and reporting issues

Feel free to create an issue in this repository if you have questions, suggestions or feature requests.

Validation, linters and pull-requests

We want to provide high quality code and modules. For this reason we are using several pre-commit hooks and GitHub Actions workflows. A pull-request to the main branch will trigger these validations and lints automatically. Please check your code before you will create pull-requests. See pre-commit documentation and GitHub Actions documentation for further details.

License

License

See LICENSE for full details.

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.