Upgrading DB version fails with InvalidParameterCombination for instance parameter group

[rosshettel]

Found a bug? Maybe our Slack Community can help.

Describe the Bug

Attempting to do a major version upgrade of an Aurora Postgres instance from 11.13 to 12.9. On latest version 50.2 and 3.63.0 of terraform. Below is my module config:

module "postgres" {
  source                      = "cloudposse/rds-cluster/aws"
  version                     = "0.50.2"
  name                        = "api-db"
  engine                      = "aurora-postgresql"
  cluster_family              = "aurora-postgresql12"
  engine_version              = "12.9"
  allow_major_version_upgrade = true
  apply_immediately           = true
  cluster_size                = 1
  admin_user                  = data.aws_ssm_parameter.db_admin_user.value
  admin_password              = data.aws_ssm_parameter.db_admin_password.value
  db_name                     = "api"
  db_port                     = 5432
  instance_type               = "db.t3.medium"
  vpc_id                      = var.vpc_id
  security_groups             = concat([aws_security_group.api.id], var.rds_security_group_inbound)
  subnets                     = var.rds_subnets
  storage_encrypted           = true
}

When running apply I get the error:

 Failed to modify RDS Cluster (api-db): InvalidParameterCombination: The current DB instance parameter group api-db-xxxxxxx is custom. You must explicitly specify a new DB instance parameter group, either default or custom, for the engine version upgrade.

Environment (please complete the following information):

Anything that will help us triage the bug will help. Here are some ideas:

• OS: OSX
• Version: 50.2
• Terraform: 3.63.0

[rosshettel]

The terraform plan output:

Terraform will perform the following actions:

  # module.api_service.module.postgres.aws_db_parameter_group.default[0] must be replaced
+/- resource "aws_db_parameter_group" "default" {
      ~ arn         = "arn:aws:rds:us-west-2:xxxxxxxxxx:pg:api-db-20220331153311430500000002" -> (known after apply)
      ~ family      = "aurora-postgresql11" -> "aurora-postgresql12" # forces replacement
      ~ id          = "api-db-20220331153311430500000002" -> (known after apply)
      ~ name        = "api-db-20220331153311430500000002" -> (known after apply)
        tags        = {
            "Name" = "api-db"
        }
        # (3 unchanged attributes hidden)
    }

  # module.api_service.module.postgres.aws_rds_cluster.primary[0] will be updated in-place
  ~ resource "aws_rds_cluster" "primary" {
      ~ allow_major_version_upgrade         = false -> true
      ~ db_cluster_parameter_group_name     = "api-db-20220331153311419900000001" -> (known after apply)
      + enable_global_write_forwarding      = false
      ~ engine_version                      = "11.13" -> "12.9"
        id                                  = "api-db"
        tags                                = {
            "Name" = "api-db"
        }
        # (33 unchanged attributes hidden)
    }

  # module.api_service.module.postgres.aws_rds_cluster_instance.default[0] will be updated in-place
  ~ resource "aws_rds_cluster_instance" "default" {
      + apply_immediately                     = true
      ~ db_parameter_group_name               = "api-db-20220331153311430500000002" -> (known after apply)
        id                                    = "api-db-1"
      ~ instance_class                        = "db.t3.large" -> "db.t3.medium"
      ~ preferred_maintenance_window          = "wed:13:04-wed:13:34" -> "wed:03:00-wed:04:00"
        tags                                  = {
            "Name" = "api-db"
        }
        # (24 unchanged attributes hidden)
    }

  # module.api_service.module.postgres.aws_rds_cluster_parameter_group.default[0] must be replaced
+/- resource "aws_rds_cluster_parameter_group" "default" {
      ~ arn         = "arn:aws:rds:us-west-2:xxxxxxxxxxx:cluster-pg:api-db-20220331153311419900000001" -> (known after apply)
      ~ family      = "aurora-postgresql11" -> "aurora-postgresql12" # forces replacement
      ~ id          = "api-db-20220331153311419900000001" -> (known after apply)
      ~ name        = "api-db-20220331153311419900000001" -> (known after apply)
        tags        = {
            "Name" = "api-db"
        }
        # (3 unchanged attributes hidden)
    }

After yes to applying the changes:

module.api_service.module.postgres.aws_db_parameter_group.default[0]: Creating...
module.api_service.module.postgres.aws_rds_cluster_parameter_group.default[0]: Creating...
module.api_service.module.postgres.aws_db_parameter_group.default[0]: Creation complete after 1s [id=api-db-20220331162335693200000001]
module.api_service.module.postgres.aws_rds_cluster_parameter_group.default[0]: Creation complete after 1s [id=api-db-20220331162335693200000002]
module.api_service.module.postgres.aws_rds_cluster.primary[0]: Modifying... [id=api-db]

And then the error:

Error: Failed to modify RDS Cluster (api-db): InvalidParameterCombination: The current DB instance parameter group api-db-20220331153311430500000002 is custom. You must explicitly specify a new DB instance parameter group, either default or custom, for the engine version upgrade.

[Gowiem]

@rosshettel what did you end up doing to get around this? Any solution you would expect to get implemented at the module layer?

[rosshettel]

@Gowiem did not end up getting around this unfortunately, had other priorities come up and so this got put in the backlog. honestly not sure what solution I'd expect since I'm still not exactly sure why this is failing. But maybe instead of using custom DB parameters it could just use the default ones? Seems like that might simplify things a bit

[woz5999]

Seems related to this, which doesn't seem too promising: aws/aws-sdk#392

[addavid]

@Gowiem This exception is thrown because the module doesn't pass a value to db_instance_parameter_group_name argument in aws_rds_cluster resource. I've opened a PR aws/aws-cli#151 that fixes this issue.