DPDK Usecase: SRIOV-CNI to revert the MAC to OriginalVfState even if not configured using it.
joelcheison opened this issue · 6 comments
What would you like to be added?
sriov-cni stores the Original VF configuration as a part of its workflow. In the case of mac address, sriov-cni only resets the mac address to the original VF state if the mac was configured using the NAD's extended configuration for sriov-cni.
In the DPDK usecase the mac address of the VF is usually set by the dpdk application inside the pod. However the application might not always get a chance to clear the mac and revert the VF to its original state, especially in cases of ungraceful pod termination.
sriov-cni always caches the Original Vf state and the Original mac address of the VF is available in the cache. During any ungraceful actions like node shutdown/reboot and the node comes back up, kubelet will call cmdDel and the cache will have the original MAC, which if sriov-cni chooses to, can replace on deletion.
What is the use case for this feature / enhancement?
In some DPDK usecases the application needs to hardcode the MAC address. DPDK application consumes the VF and sets the mac address. On an ungraceful deletion, the pods get recreated and consumes a different VF. As the mac is hardcoded, the new VF comes up with the same mac and due to the ungraceful termination the application could not handle the mac deletion.
Even though sriov-cni has the original mac of the VF, with current design it does not reset the mac as the mac was not set using sriov-cni.
Making the following change, sriov-cni can reset the MAC and aid such usecases:
Patch_mac_reset.diff.txt
Hi @joelcheison if the mac address is hardcoded why not let the sriov-cni always configure and revert it?
example in the pod definition
apiVersion: v1
kind: Pod
metadata:
annotations:
k8s.v1.cni.cncf.io/networks: '[
{
"name": "dpdk-network-1",
"mac": "60:00:00:00:00:03",
"namespace": "seba"
},
]'
labels:
app: dpdk
name: testpmd-1
namespace: seba
If the mac is set in the NAD, then one NAD needs to be created for each pod. Whereas if we configure parameters like VLAN, spoofchk etc, one NAD can serve multiple pods. And a set of pods that hardcodes one mac address for each pod could still use a single NAD.
How you configure the mac address on every pod?
The example I gave you is a pod definition not a NAD definition
Hi @SchSeba ,
It is okay to use for a single pod definition. How can we use this if we are using a deployment with 10 replica's?
If you give a single mac address in the pod template of the deployment definition, all 10 pods would come up with the same MAC address right?
Hi @joelcheison for something like that you can have a look at https://github.com/k8snetworkplumbingwg/kubemacpool
closing this issue for housekeeping feel free to reopen if needed