aws-ia/terraform-aws-eks-blueprints-teams

bug: Error namespace not found

csantanapr opened this issue · 0 comments

When creating resources that are namespaced like resourcequotas, limitranges, roles, network policy, service accounts and the namespace is not created first you will encounter an error because the namespace is not found

│ Error: namespaces "backend-frontend" not found
│ 
│   with module.spoke_cluster.module.app_teams["frontend"].kubernetes_role_binding_v1.this["backend-frontend"],
│   on .terraform/modules/spoke_cluster.app_teams/main.tf line 344, in resource "kubernetes_role_binding_v1" "this":
│  344: resource "kubernetes_role_binding_v1" "this" {

I think (but I'm not 100% sure) the root cause is a race condition namespaces been created in parallel as also the other resources

We could add a depends_on, or we could iterated over the map kubernetes_namespace_v1.this instead of var.namespaces
like in networkpolicy in this case:

resource "kubernetes_network_policy_v1" "this" {
  for_each = { for k, v in var.namespaces : k => v if try(v.create, true) && length(try(v.network_policy, {})) > 0 }

We could do a retry, but I don't see a apply_retry_count option for kubernetes terraform provider like the the kubectl terraform provider

Here is the example I was trying:

module "app_teams" {
  source = "github.com/aws-ia/terraform-aws-eks-blueprints-teams"

  for_each = {
    frontend = {}
    crystal  = {}
    nodejs   = {}
  }
  name = "app-team-${each.key}"


  users             = [data.aws_caller_identity.current.arn]
  cluster_arn       = module.eks.cluster_arn
  oidc_provider_arn = module.eks.oidc_provider_arn

  namespaces = {

    "backend-${each.key}" = {
      create_service_account = false

      labels = {
        appName     = "eks-teams-app",
        projectName = "project--eks-blueprints",
      }

      resource_quota = {
        hard = {
          "limits.cpu"      = "4",
          "limits.memory"   = "16Gi",
          "requests.cpu"    = "2",
          "requests.memory" = "4Gi",
          "pods"            = "20",
          "secrets"         = "20",
          "services"        = "20"
        }
      }
      limit_range = {
        limit = [
          {
            type = "Pod"
            max = {
              cpu    = "2"
              memory = "1Gi"
            }
          },
          {
            type = "Container"
            default = {
              cpu    = "500m"
              memory = "512Mi"
            }
            default_request = {
              cpu    = "100m"
              memory = "128Mi"
            }
          }
        ]
      }
    }
  }


  tags = local.tags
}