CiscoDevNet/terraform-provider-nxos

After removing resource cdp/lldp config left on interface

Closed this issue · 8 comments

krzysztofmaciejewskiit commented

When I delete a part of the code in e.g. part two, responsible for cdp or lldp it says it is destroyed, but it stays on the Nexus configuration and I don't know why all other things get deleted and those 3 lines of code (seen on the switch) stay, even though the terraform logs clearly say "destroyed". My removal is to simply remove the part of the code responsible for adding cdp and lldp to the second interface, that is, everything from that tag down:
//============================================== part-2 ==============================================

image
image
image

Code from terraform: main.tf

terraform {
  required_providers {
    nxos = {
      source  = "CiscoDevNet/nxos"
      version = "0.5.3"
    }
  }
}

provider "nxos" {
  username = var.nxos_username
  password = var.nxos_password
  url      = var.nxos_url
}

/*============================================ common ============================================*/

resource "nxos_feature_lldp" "lldp" {
  admin_state = "enabled"
}

/*=====================================*/

resource "nxos_rest" "cdpEntity" {
  dn         = "sys/cdp"
  class_name = "cdpEntity"
}

//============================================== part-1 ==============================================
//============================================== part-1 ==============================================
//============================================== part-1 ==============================================

/*============================================ description & L3 ============================================*/

resource "nxos_physical_interface" "desc-L3" {
  interface_id = "eth1/5"
  description  = "desc1"
  layer = "Layer3"
  admin_state  = "up"
  user_configured_flags = "admin_state"
}

/*============================================ lldp ============================================*/

resource "nxos_rest" "lldpInst" {
  depends_on = [nxos_feature_lldp.lldp]
  dn         = "sys/lldp/inst"
  class_name = "lldpInst"
  children = [
    {
      rn         = "if-[eth1/5]"
      class_name = "lldpIf"
      content = {
        adminRxSt = "disabled",
        adminTxSt = "disabled",
        id        = "eth1/5"
      }
    }
  ]
}

/*============================================ cdp ============================================*/

resource "nxos_rest" "cdpInst" {
  depends_on = [nxos_rest.cdpEntity]
  dn         = "sys/cdp/inst"
  class_name = "cdpInst"
  children = [
    {
      rn         = "if-[eth1/5]"
      class_name = "cdpIf"
      content = {
        adminSt = "disabled",
        id        = "eth1/5"
      }
    }
  ]
}

//============================================== part-2 ==============================================
//============================================== part-2 ==============================================
//============================================== part-2 ==============================================

/*============================================ description & L3 ============================================*/

resource "nxos_physical_interface" "desc-L3v2" {
  interface_id = "eth1/6"
  description = "desc2"
  layer = "Layer3"
  admin_state  = "up"
  user_configured_flags = "admin_state"
}

/*============================================ lldp ============================================*/

resource "nxos_rest" "lldpInstv2" {
  depends_on = [nxos_feature_lldp.lldp]
  dn         = "sys/lldp/inst"
  class_name = "lldpInst"
  children = [
    {
      rn         = "if-[eth1/6]"
      class_name = "lldpIf"
      content = {
        adminRxSt = "disabled",
        adminTxSt = "disabled",
        id        = "eth1/6"
      }
    }
  ]
}

/*============================================ cdp ============================================*/

resource "nxos_rest" "cdpInstv2" {
  depends_on = [nxos_rest.cdpEntity]
  dn         = "sys/cdp/inst"
  class_name = "cdpInst"
  children = [
    {
      rn         = "if-[eth1/6]"
      class_name = "cdpIf"
      content = {
        adminSt = "disabled",
        id        = "eth1/6"
      }
    }
  ]
}

Link to Cisco Community discussion: https://community.cisco.com/t5/devnet-general-discussions/terraform-after-removing-resource-cdp-lldp-config-left-on/td-p/5157851

danischm commented

There is no need to manage the "sys/lldp/inst" and "sys/cdp/inst" objects using Terraform. These are system level objects that already exist and therefore there is no need to manage them using Terraform. What happens here is, TF attempts to delete those system objects (which would implicitly also delete the child objects), but it fails as those types of objects cannot be deleted and therefore the child objects remain in place as well. The "nxos_rest" resource silently suppresses this error, because it is a generic resource and in some cases this is expected. The resources should therefore be defined like this:

resource "nxos_rest" "lldpIf" {
  depends_on = [nxos_feature_lldp.lldp]
  dn         = "sys/lldp/inst/if-[eth1/5]"
  class_name = "lldpIf"
  content = {
    adminRxSt = "disabled"
    adminTxSt = "disabled"
    id        = "eth1/5"
  }
}

resource "nxos_rest" "cdpIf" {
  depends_on = [nxos_rest.cdpEntity]
  dn         = "sys/cdp/inst/if-[eth1/5]"
  class_name = "cdpIf"
  content = {
    adminSt = "disabled"
    id        = "eth1/5"
  }
}
krzysztofmaciejewskiit commented

What you propose does not work.
This is a very reduced version of my code. It configures the following interfaces and on each one I want to enter

  no lldp transmit
  no lldp receive
  no cdp enable

so here globally I want to ‘enable’ cdp and lldp.

resource "nxos_feature_lldp" "lldp" {
  admin_state = "enabled"
}

resource "nxos_rest" "cdpEntity" {
  dn         = "sys/cdp"
  class_name = "cdpEntity"
}

Then on each subsequent interface I want to use ‘depends on’ so that's why I want to ‘enable cdp and lldp globally’. If I ever need to remove the configuration from an interface, then, I would simply remove the part of the code in Terraform that previously allowed it to be configured. I would then expect the commands (no lldp transmit, no lldp receive, no cdp enable) to be removed, but this does not happen.

Even after removing this part, my interface looks like this:

interface Ethernet1/3
  no lldp transmit
  no lldp receive
  no cdp enable

My entire code:

terraform {
  required_providers {
    nxos = {
      source  = "CiscoDevNet/nxos"
      version = "0.5.5"
    }
  }
}

provider "nxos" {
  username = var.nxos_username
  password = var.nxos_password
  url      = var.nxos_url
}

/*========== COMMON ==========*/

resource "nxos_feature_lldp" "lldp" {
  admin_state = "enabled"
}

resource "nxos_rest" "cdpEntity" {
  dn         = "sys/cdp"
  class_name = "cdpEntity"
}

/*========== LLDP ==========*/

resource "nxos_rest" "lldpIf" {
  dn         = "sys/lldp/inst/if-[${var.interface}]"
  class_name = "lldpIf"
  content     = {
    adminRxSt = "disabled",
    adminTxSt = "disabled",
    id        = var.interface
  }
}

/*========== CDP ==========*/

resource "nxos_rest" "cdpIf" {
  dn         = "sys/cdp/inst/if-[${var.interface}]"
  class_name = "cdpIf"
    content  = {
    adminSt  = "disabled",
    id       = var.interface
  }
}

I would expect that when my whole code looks like this (after removing the interface configuration):

terraform {
  required_providers {
    nxos = {
      source  = "CiscoDevNet/nxos"
      version = "0.5.5"
    }
  }
}

provider "nxos" {
  username = var.nxos_username
  password = var.nxos_password
  url      = var.nxos_url
}

/*========== COMMON ==========*/

resource "nxos_feature_lldp" "lldp" {
  admin_state = "enabled"
}

resource "nxos_rest" "cdpEntity" {
  dn         = "sys/cdp"
  class_name = "cdpEntity"
}

Then on the interface on the switch, the configuration should be cleared of all configuration, and as I say, the configuration stays.

danischm commented

The cdpIf object cannot be deleted and therefore the configuration does not get removed on the CLI either. You can verify that by enabling debug logging where you can see the following message:

[DEBUG] HTTP Response: {"imdata":[{"error": {"attributes": {"code": "107","text": "Cannot delete object of class:cdpIf"}}}]}

This is how the DME model behaves and there is nothing really we can do on the Terraform side.

krzysztofmaciejewskiit commented

Why can I add via terraform ‘no cdp enable’ but cannot withdraw this change? Doesn't it make no sense that I can configure this on the interface but can't remove it?
On the other hand, I can remove the LLDP when I delete the whole thing this way (the LLDP configuration then disappears from the interface):

resource "nxos_feature_lldp" "lldp" {
  admin_state = "enabled"
}

resource "nxos_rest" "lldpIf" {
  dn         = "sys/lldp/inst/if-[${var.interface}]"
  class_name = "lldpIf"
  content     = {
    adminRxSt = "disabled",
    adminTxSt = "disabled",
    id        = var.interface
  }
}

However, this also makes no sense (deleting the whole thing), as other interfaces that have LLDP configured need this feature enabled

danischm commented

You can explicitly set the adminSt to enabled which will re-enable CDP and remove the respective CLI command, but you cannot re-enable it by deleting/destroying the resource, because the DME model does not support that.

krzysztofmaciejewskiit commented

Unfortunately this doesn't satisfy me, because the approach we are taking in the project involves removing some code from the TF if we want to remove the configuration on the interface. But ok, we will get around the problem somehow.

  • However, it makes me wonder why, some things have dedicated resources and other things have to be done via nxos_rest?
  • Why the LLDP feature is enabled with a dedicated resource, then you have to use nxos_rest to configure LLDP on the interface?
  • Why is there no dedicated resource to run CDP, but there is for LLDP?
  • In this particular case, I wonder why lldpIf I can remove and cdpIf not.
    Can you explain this to me? I would appreciate it.
danischm commented
  • However, it makes me wonder why, some things have dedicated resources and other things have to be done via nxos_rest?
  • Why the LLDP feature is enabled with a dedicated resource, then you have to use nxos_rest to configure LLDP on the interface?
  • Why is there no dedicated resource to run CDP, but there is for LLDP?

Simply because we haven't implemented dedicated resources for all the objects yet. In case a specific resource doesn't exist yet, we can use the generic nxos_rest resource as a workaround until we have implemented a dedicated resource. We have implemented a framework allowing us to add support for new resources without writing any code and are also open for community contributions.

In this particular case, I wonder why lldpIf I can remove and cdpIf not. Can you explain this to me? I would appreciate it.

That is indeed surprising, but apparently NX-API/DME allows deleting an lldpIf object while the same is not allowed for cdpIf objects. You can use Visore (https://developer.cisco.com/docs/nx-os/visore/) to verify this yourself.

krzysztofmaciejewskiit commented

Ok thanks.
I use this visore tool when I need to write my resource.
I wish for myself and you that more resources are created and faster, because TF is very cool.