Incorrect handling of NLB forwarding rule targets

[benschmi]

Description

The targets of a network load balancer forwarding rules are not always returned in the same order by the Cloud API. The Terraform provider does not handle this correctly. On every subsequent apply, TF will want to update (in-place) all target IPs.

Expected behavior

The order is which the Cloud API returns the targets should be irrelevant. It should not trigger replace of this resource.

Environment

Terraform version:

Terraform v1.1.5
on darwin_amd64
+ provider registry.terraform.io/ionos-cloud/ionoscloud v6.1.3

Configuration Files

///////////////////////////////////////////////////////////
// Network Load Balancer
///////////////////////////////////////////////////////////

resource "ionoscloud_networkloadbalancer" "nlb" {
  datacenter_id = ionoscloud_datacenter.publicK8s.id
  name          = "nlb"
  listener_lan  = ionoscloud_lan.public_lan.id
  target_lan    = ionoscloud_lan.private_lan.id
  ips           = [ionoscloud_ipblock.nlb_IP.ips[0]]
}

data "ionoscloud_networkloadbalancer" "nlb" {
  datacenter_id = ionoscloud_datacenter.publicK8s.id
  name            = "nlb"
}

///////////////////////////////////////////////////////////
// NLB Forwarding Rule
///////////////////////////////////////////////////////////

resource "ionoscloud_networkloadbalancer_forwardingrule" "nlb_fr_01" {
 datacenter_id = ionoscloud_datacenter.publicK8s.id
 networkloadbalancer_id = ionoscloud_networkloadbalancer.nlb.id
 name = "fr-01"
 algorithm = "RANDOM"
 protocol = "TCP"
 listener_ip = ionoscloud_ipblock.nlb_IP.ips[0]
 listener_port = "80"
 dynamic "targets" {
     for_each = var.IPs

     content {
        ip = cidrhost(data.ionoscloud_networkloadbalancer.nlb.lb_private_ips[0], targets.value)
        port = "31234"
        weight = "1"
        health_check {
            check = true
            check_interval = 1000
        }
     }
 }
}

var.IPs contains all possible IPs (11-240).

How to Reproduce

1. Create NLB and forwarding rules with above config
2. terraform apply
3. terraform apply again

Error and Debug Output

# ionoscloud_networkloadbalancer_forwardingrule.nlb_fr_01 will be updated in-place
  ~ resource "ionoscloud_networkloadbalancer_forwardingrule" "nlb_fr_01" {
        id                     = "a6f4be09-eb5f-49fb-b951-ec84f71ecf8e"
        name                   = "fr-01"
        # (6 unchanged attributes hidden)


      ~ targets {
          ~ ip     = "10.7.222.100" -> "10.7.222.11"
            # (2 unchanged attributes hidden)

            # (1 unchanged block hidden)
        }
      ~ targets {
          ~ ip     = "10.7.222.101" -> "10.7.222.12"
            # (2 unchanged attributes hidden)

Additional Notes

This also happens every now and then to the datacenter features:

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # ionoscloud_datacenter.publicK8s has changed
  ~ resource "ionoscloud_datacenter" "publicK8s" {
      ~ features            = [
            # (5 unchanged elements hidden)
            "k8s",
          - "vnf-fw",
            "ssd-storage-zoning",
          + "vnf-fw",
            "vnf-lb",
            # (15 unchanged elements hidden)