Apply results in an incorrect Update for TypeSet with an unexpected empty element

[prashantv]

SDK version

github.com/hashicorp/terraform-plugin-sdk/v2 v2.10.1

Relevant provider source code

Repo with a full provider + repro instructions: https://github.com/prashantv/terraform-set-null-repro

Relevant provider code:
https://github.com/prashantv/terraform-set-null-repro/blob/main/internal/provider/service_resource.go

Terraform Configuration Files

This configuration refers to a dummy resource that was created to repro the issue without any external dependencies.
See https://github.com/prashantv/terraform-set-null-repro for details.

To reproduce the issue, namespace is changed to some other value, and then applied. The apply results in an unexpected job set that contains 2 elements, where one is empty.

resource "tftest_service" "s" {
  name    = "svc1"
  job {
    namespace = "NS1"
    search_tags = {
      "foo" = "bar"
    }
  }
}
...

Debug Output

https://gist.github.com/prashantv/b7847ee5e35fa0167b7580cd10598faf

Expected Behavior

Apply to send the provider a single element in the "job" set.

Actual Behavior

Apply causes an Update that has 2 elements in the "job" set, where the first is empty, formatting the panic message:

panic: jobSet has more elements than expected: [
  map[namespace: search_tags:map[]]    ## unexpected empty element in set
  map[namespace:NS2 search_tags:map[foo:bar]]
]

Steps to Reproduce

To reproduce this issue, a repository is created with a specific dummy provider (that has no external dependencies), and with exact repro steps,
https://github.com/prashantv/terraform-set-null-repro

The custom provider is required, as this issue requires a specific setup:

• TypeSet field which contains a nested schema (rather than just a simple primitive like string)
• In the nested schema, one of the fields is a TypeMap, and the TypeMap has values specified in the .tf file

References

Originally filed on the terraform repo, but directed to the SDK,
hashicorp/terraform#30581

Additional Information

In case it helps anyone else, we mitigated this issue by using a TypeList instead of TypeSet, so a field change doesn't result in the whole object changing, which seems to avoid the specific bug.

[disaster37]

Same matter for me

[ewbankkit]

Similar: #588.

[bflad]

Hi @prashantv 👋 Thank you for raising this and sorry you ran into trouble here. This is certainly unexpected behavior.

It appears this is a similar class of problem as #652, so to consolidate efforts and discussions, I'm going to close this issue in preference of the existing one. Please follow (and potentially 👍 react to) that issue for further updates.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.