nmstate/kubernetes-nmstate

How to define ip list to be desired, but not the complete list?

Opened this issue · 2 comments

nabbas-ca commented

What happened:
I created an nncp policy that configures a bond on the openshift provisioning network, with 2 static ipv6 addresses. Adding bonding interface , with set of ipv6 static ips, conflicts with openshift extra VIP it configures on the bond on one of the master nodes, making the nncp policy fail.
What you expected to happen:
nncp policy to successfully configure, even if not the desired list of ips is not the full list. I need the policy to ensure the set of ips to be present, but not exclusive. If there is an extra ip, it is ok.
How to reproduce it (as minimally and precisely as possible):
Create the following nncp on openshift cluster with bonding is configured on the provisioning network

apiVersion: nmstate.io/v1beta1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: bond1-control-2-policy
spec:
  desiredState:
    interfaces:
    - description: Bond connection enslaving ens3f0 and ens3f1
      ipv4:
        enabled: false
      ipv6:
        address:
        - ip: fd8c:215d:15ee:c000:63f:72ff:fef5:509e
          prefix-length: 64
        - ip: fe80::63f:72ff:fef5:509e
          prefix-length: 64
        autoconf: false
        dhcp: false
        enabled: true
      link-aggregation:
        mode: 802.3ad
        options:
          lacp_rate: "1"
          miimon: "140"
          xmit_hash_policy: "1"
        slaves:
        - ens3f0
        - ens3f1
      mtu: 9000
      name: bond1
      state: up
      type: bond
  nodeSelector:
    kubernetes.io/hostname: control-2.cluster1.example.com

Anything else we need to know?:
Wondering if there is a supported way to just add ips, and not make sure the list of IPs are the only IPs present.
Environment:

  • NodeNetworkState on affected nodes (use kubectl get nodenetworkstate <node_name> -o yaml): Not necessary, i can provide if neeeded
  • Problematic NodeNetworkConfigurationPolicy: see above
  • kubernetes-nmstate image (use kubectl get pods --all-namespaces -l app=kubernetes-nmstate -o jsonpath='{.items[0].spec.containers[0].image}'): registry.redhat.io/container-native-virtualization/kubernetes-nmstate-handler-rhel8@sha256:8853dc6495d723c582a52a519d13bc188883797a900240586fac442492431d08
  • NetworkManager version (use nmcli --version) nmcli tool, version 1.30.0-10.el8_4
  • Kubernetes version (use kubectl version):
    Client Version: 4.7.16
    Server Version: 4.8.19
    Kubernetes Version: v1.21.4+6438632
  • OS (e.g. from /etc/os-release):
  • sh-4.4# cat /etc/os-release
    NAME="Red Hat Enterprise Linux CoreOS"
    VERSION="48.84.202110270303-0"
    ID="rhcos"
    ID_LIKE="rhel fedora"
    VERSION_ID="4.8"
    PLATFORM_ID="platform:el8"
    PRETTY_NAME="Red Hat Enterprise Linux CoreOS 48.84.202110270303-0 (Ootpa)"
    ANSI_COLOR="0;31"
    CPE_NAME="cpe:/o:redhat:enterprise_linux:8::coreos"
    HOME_URL="https://www.redhat.com/"
    DOCUMENTATION_URL="https://docs.openshift.com/container-platform/4.8/"
    BUG_REPORT_URL="https://bugzilla.redhat.com/"
    REDHAT_BUGZILLA_PRODUCT="OpenShift Container Platform"
    REDHAT_BUGZILLA_PRODUCT_VERSION="4.8"
    REDHAT_SUPPORT_PRODUCT="OpenShift Container Platform"
    REDHAT_SUPPORT_PRODUCT_VERSION="4.8"
    OPENSHIFT_VERSION="4.8"
    RHEL_VERSION="8.4"
    OSTREE_VERSION='48.84.202110270303-0'
  • Others:
qinqon commented

@nabbas-ca can we close #727 ? it looks like an old verision of this issue.

qinqon commented

At newer kubernetes-nmstate (OCP 4.10) we have a capture feature to copy stuff from current state, if the number of "extra" ips is know we can use capture to copy them as part of the NNCP

apiVersion: nmstate.io/v1beta1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: bond1-control-2-policy
spec:
  capture: 
    bond1-iface: interfaces.name == "bond1"
  desiredState:
    interfaces:
    - description: Bond connection enslaving ens3f0 and ens3f1
      ipv4:
        enabled: false
      ipv6:
        address:
        - ip: "{{ capture.bond1-iface.interfaces.0.ipv6.address.0.ip }}"
          prefix-length: "{{ capture.bond1-iface.interfaces.0.ipv6.address.0.prefix-length }}"
        - ip: fd8c:215d:15ee:c000:63f:72ff:fef5:509e
          prefix-length: 64
        - ip: fe80::63f:72ff:fef5:509e
          prefix-length: 64
        autoconf: false
        dhcp: false
        enabled: true
      link-aggregation:
        mode: 802.3ad
        options:
          lacp_rate: "1"
          miimon: "140"
          xmit_hash_policy: "1"
        slaves:
        - ens3f0
        - ens3f1
      mtu: 9000
      name: bond1
      state: up
      type: bond
  nodeSelector:
    kubernetes.io/hostname: control-2.cluster1.example.com