Add configuration for device flags
bjosv opened this issue · 12 comments
What would you like to be added?
A configuration parameter for setting device flags.
Like the IFF_ALLMULTI flag to enable propagation of multicast packets from the network.
What is the use case for this feature / enhancement?
One example would be when using IPv6 and having Neighbor Discovery Protocol (NDP) implemented in a Network Functions (NF) userland stack. The NF then needs to be able to receive multicast Ethernet packets which requires setting the flag IFF_ALLMULTI on the device to Receive all multicast packets.
The NF should not set this flag itself because it requires CAP_NET_ADMIN capability which normally is considered too wide and insecure. Having the CNI do the scaffolding would be more proper.
@bjosv Hey, we briefly discussed this at the additional community meeting. We do not know of any CNI that we can 'chain' that can accomplish this task instead of us adding this functionality. @zshi-redhat do you know of any?
Just so I understand the ask correctly - what you're asking for is equivalent to
ip link set $VF allmulticast on
I know netlink has an API for this. We could easily accomplish this in SetupVFfunction with our netlink VF object .
Also, we want to know if there is a dependency on any PF configuration when enabling this for a VF? I do not believe so. Can you confirm?
@bjosv Hey, we briefly discussed this at the additional community meeting. We do not know of any CNI that we can 'chain' that can accomplish this task instead of us adding this functionality. @zshi-redhat do you know of any?
I'm not ware of any chain CNI that can do this job.
Just so I understand the ask correctly - what you're asking for is equivalent to
ip link set $VF allmulticast onI know netlink has an API for this. We could easily accomplish this in
SetupVFfunction with our netlink VF object .
+1
I thought about this further since yesterday's meeting,
If the configuration applies to a netdevice in general,
Perhaps tuning CNI plugin can be extended with this functionality. WDYT?
https://www.cni.dev/plugins/current/meta/tuning/
https://github.com/containernetworking/plugins/tree/master/plugins/meta/tuning
Hi, thanks for taking time to discuss this!
Just so I understand the ask correctly - what you're asking for is equivalent to
ip link set $VF allmulticast on
Yes, this would set the same flag on the device.
Also, we want to know if there is a dependency on any PF configuration when enabling this for a VF? I do not believe so. Can you confirm?
No, there is no dependency what we have seen.
If the configuration applies to a netdevice in general,
Perhaps tuning CNI plugin can be extended with this functionality. WDYT?
Good point, this would probably be needed for other CNIs aswell.
Maybe this would be a better solution.
Are there any common rules, or collaborations between CNIs communities, where to put features like this?
Asking since we seem to have the possiblity to change mac (config mac) in both sr-iov and tuning.
Maybe this was introduced before chaning existed in the CNI spec?
If the configuration applies to a netdevice in general,
Perhaps tuning CNI plugin can be extended with this functionality. WDYT?
With tuning CNI plugin, how can we add the sysctl path that requires knowing of container interface name?
Good point, this would probably be needed for other CNIs aswell.
Maybe this would be a better solution.Are there any common rules, or collaborations between CNIs communities, where to put features like this?
My understanding of current approach is to re-use what's exist in meta plugins for sriov-cni where appropriate.
Asking since we seem to have the possiblity to change mac (config
mac) in both sr-iov and tuning.
MAC is a little special as we have different implementation in sriov-cni (the admin and effective MACs).
With tuning CNI plugin, how can we add the sysctl path that requires knowing of container interface name?
Im not sure i understand the issue, can you elaborate ?
With tuning CNI plugin, how can we add the sysctl path that requires knowing of container interface name?
Im not sure i understand the issue, can you elaborate ?
For example, to enable arp_notify for a VF device, it sets the following sys path:
/proc/sys/net/ipv4/conf/<VF-NAME>/arp_notify
The is the interface name inside container namespace, it is assigned by Multus and cannot be pre-determined when creating tuning CNI net-attach-def.
I dont understand how it differs from setting mac address for an interface inside container via tuning CNI plugin.
the container interface name is provided by the runtime, see : https://github.com/containernetworking/plugins/blob/e27c48b391539f5536918b9c379c59ef2793cb0d/plugins/meta/tuning/tuning.go#L320
The tuning CNI config would need to have arp_notify=true or similar then use the provided args.Ifname to set it.
Am i missing something ?
I dont understand how it differs from setting mac address for an interface inside container via tuning CNI plugin.
the container interface name is provided by the runtime, see : https://github.com/containernetworking/plugins/blob/e27c48b391539f5536918b9c379c59ef2793cb0d/plugins/meta/tuning/tuning.go#L320The tuning CNI config would need to have
arp_notify=trueor similar then use the providedargs.Ifnameto set it.Am i missing something ?
Okay, I was thinking we need to specify the full sys path in tuning net-attach-def, which is not the case for MAC/PROMISC etc. so it can be implemented by referring to the args.IfName passed down from previous plugin.
Okay, I was thinking we need to specify the full sys path in tuning net-attach-def, which is not the case for MAC/PROMISC etc. so it can be implemented by referring to the args.IfName passed down from previous plugin.
Thanks. I have done some minimal testing now by adding the flag to tuning (using netlink and args.IfName) and it seemed to trigger the change atleast when checking with ip link within the container.
I would guess I only need to set the allmulti-flag on the device in the container when using sr-iov as the main CNI.
Maybe there are other CNIs that would need other handling, like doing something on a created OVS bridge. But I guess I have to figure that out..
Have I understood it correctly that you would run the sriov-cni as the first/main CNI using multus, and having tuning receiving the "prevResult" from sriov-cni, which it only pass-through in the allmulticast case? There is no way of knowing in a plugin that previous plugins have already changed flags/parameters, and the last plugin have the final say I guess?
..i.e the multus configuration will by its order and setting decide where a given parameter is managed.
Have I understood it correctly that you would run the sriov-cni as the first/main CNI using multus, and having tuning receiving the "prevResult" from sriov-cni, which it only pass-through in the allmulticast case
Yes, that is correct, however i dont know in this case you will need the prevResult as a cni ADD call will contain the container ifname in its args. see https://github.com/containernetworking/cni/blob/master/SPEC.md#add-add-container-to-network-or-apply-modifications
There is no way of knowing in a plugin that previous plugins have already changed flags/parameters, and the last plugin have the final say I guess
you could know by checking the status of the interface before changing it e.g if it changed from default.
multus configuration will by its order and setting decide where a given parameter is managed.
yes, multus is expected to execute the plugins in the network configuration by order. it is the responsibility of the "admin"
to define a coherent configuration (e.g not change a certain parameter/config in multiple places by multiple cnis in the list of plugins).
Closing this issue in favor of containernetworking/plugins#621 and containernetworking/plugins#624.
Thanks for all advice.