SR-IOV CNI plugin

SR-IOV CNI plugin

This plugin enables the configuration and usage of SR-IOV VF networks in containers and orchestrators like Kubernetes.

Network Interface Cards (NICs) with SR-IOV capabilities are managed through physical functions (PFs) and virtual functions (VFs). A PF is used by the host and usually represents a single NIC port. VF configurations are applied through the PF. With SR-IOV CNI each VF can be treated as a separate network interface, assigned to a container, and configured with it's own MAC, VLAN, IP and more.

SR-IOV CNI plugin works with SR-IOV device plugin for VF allocation in Kubernetes. A metaplugin such as Multus gets the allocated VF's deviceID(PCI address) and is responsible for invoking the SR-IOV CNI plugin with that deviceID.

Build

This plugin uses Go modules for dependency management and requires Go 1.17+ to build.

To build the plugin binary:

make

Upon successful build the plugin binary will be available in build/sriov.

Kubernetes Quick Start

A full guide on orchestrating SR-IOV virtual functions in Kubernetes can be found at the SR-IOV Device Plugin project.

Creating VFs is outside the scope of the SR-IOV CNI plugin. More information about allocating VFs on different NICs can be found here

To deploy SR-IOV CNI by itself on a Kubernetes 1.16+ cluster:

kubectl apply -f images/sriov-cni-daemonset.yaml

Note The above deployment is not sufficient to manage and configure SR-IOV virtual functions. See the full orchestration guide for more information.

Usage

SR-IOV CNI networks are commonly configured using Multus and SR-IOV Device Plugin using Network Attachment Definitions. More information about configuring Kubernetes networks using this pattern can be found in the Multus configuration reference document.

A Network Attachment Definition for SR-IOV CNI takes the form:

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: sriov-net1
  annotations:
    k8s.v1.cni.cncf.io/resourceName: intel.com/intel_sriov_netdevice
spec:
  config: '{
  "type": "sriov",
  "cniVersion": "0.3.1",
  "name": "sriov-network",
  "ipam": {
    "type": "host-local",
    "subnet": "10.56.217.0/24",
    "routes": [{
      "dst": "0.0.0.0/0"
    }],
    "gateway": "10.56.217.1"
  }
}'

The .spec.config field contains the configuration information used by the SR-IOV CNI.

Basic configuration parameters

The following parameters are generic parameters which are not specific to the SR-IOV CNI configuration, though (with the exception of ipam) they need to be included in the config.

cniVersion : the version of the CNI spec used.
type : CNI plugin used. "sriov" corresponds to SR-IOV CNI.
name : the name of the network created.
ipam (optional) : the configuration of the IP Address Management plugin. Required to designate an IP for a kernel interface.

Example configurations

The following examples show the config needed to set up basic SR-IOV networking in a container. Each of the json config objects below can be placed in the .spec.config field of a Network Attachment Definition to integrate with Multus.

Kernel driver config

This is the minimum configuration for a working kernel driver interface using an SR-IOV Virtual Function. It applies an IP address using the host-local IPAM plugin in the range of the subnet provided.

{
  "type": "sriov",
  "cniVersion": "0.3.1",
  "name": "sriov-network",
  "ipam": {
    "type": "host-local",
    "subnet": "10.56.217.0/24",
    "routes": [{
      "dst": "0.0.0.0/0"
    }],
    "gateway": "10.56.217.1"
  }
}

Extended kernel driver config

This configuration sets a number of extra parameters that may be key for SR-IOV networks including a vlan tag, disabled spoof checking and enabled trust mode. These parameters are commonly set in more advanced SR-IOV VF based networks.

{
  "cniVersion": "0.3.1",
  "name": "sriov-advanced",
  "type": "sriov",
  "vlan": 1000,
  "spoofchk": "off",
  "trust": "on",
  "ipam": {
    "type": "host-local",
    "subnet": "10.56.217.0/24",
    "routes": [{
      "dst": "0.0.0.0/0"
    }],
    "gateway": "10.56.217.1"
  }
}

DPDK userspace driver config

The below config will configure a VF using a userspace driver (uio/vfio) for use in a container. If this plugin is used with a VF bound to a dpdk driver then the IPAM configuration will still be respected, but it will only allocate IP address(es) using the specified IPAM plugin, not apply the IP address(es) to container interface. Other config parameters should be applicable but implementation may be driver specific.

{
    "cniVersion": "0.3.1",
    "name": "sriov-dpdk",
    "type": "sriov",
    "vlan": 1000
}

Note DHCP IPAM plugin can not be used for VF bound to a dpdk driver (uio/vfio).

Note When VLAN is not specified in the Network-Attachment-Definition, or when it is given a value of 0, VFs connected to this network will have no vlan tag.

Advanced Configuration

SR-IOV CNI allows the setting of other SR-IOV options such as link-state and quality of service parameters. To learn more about how these parameters are set consult the SR-IOV CNI configuration reference guide

Contributing

To report a bug or request a feature, open an issue on this repo using one of the available templates.

nokia/sriov-cni