CNI delegation failed due to error:Error delegating ADD to CNI plugin:sriov because:OS exec call faild:netplugin failed with no error message
Opened this issue · 2 comments
Is this a BUG REPORT or FEATURE REQUEST?:
bug
What happened:
We have sriov-o1c-host0 cluster network and 4 pods are using that CNI.
1st pod { "clusterNetwork": "sriov-o1c-host0", "ip6": "fdfb:6442:1:673::1", "proutes6": {"::0/0": "fdfb:6442:1:673::7fff"} },
2nd pod { "clusterNetwork": "sriov-o1c-host0", "ip6": "fdfb:6442:1:673::2", "proutes6": {"::0/0": "fdfb:6442:1:673::7fff"} },
3rd pod { "clusterNetwork": "sriov-o1c-host0", "ip": "10.0.106.65", "proutes": {"11.0.0.0/20": "10.0.106.72"} }
4th pod {"clusterNetwork": "sriov-o1c-host0", "ip": "10.0.106.72", "ip6": "fdfb:6442:1:673::7fff" },But when we deploy pods one pod is in container creating state , and when pod is described below error is observed.
Warning FailedCreatePodSandBox 30m kubelet, controller-0 Failed to create pod sandbox: rpc error: code = Unknown desc = faled to setup network for sandbox "cd96d7ad7726d3ff73b4779eb98344eae19c95791cc26583652c5709922af75e": CNI network could not be set up: CNI operaton for network:sriov-o1c-host0 failed with:CNI delegation failed due to error:Error delegating ADD to CNI plugin:sriov because:OS exec call faild:netplugin failed with no error message
What you expected to happen:
All the pods should be up and running
How to reproduce it:
Deploy pods with cluster network as shown above. One pod will not come up.
This occurs sometimes
Anything else we need to know?:
Danm cleaner pods are running in the setup
kubectl get pods -A | grep -i danm-cleaner
kube-system danm-cleaner-5dtgr 1/1 Running 0 99m
Environment:
- DANM version (use
danm -version
):
/usr/libexec/cni/danm --version
2021/11/24 05:33:55 DANM binary was built from release: v4.2.1
2021/11/24 05:33:55 DANM binary was built from commit: abd3c48d_dirty
- Kubernetes version (use
kubectl version
):
kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.1", GitCommit:"7879fc12a63337efff607952a323df90cdc7a335", GitTreeState:"archive", BuildDate:"2021-06-10T00:00:03Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.1", GitCommit:"7879fc12a63337efff607952a323df90cdc7a335", GitTreeState:"clean", BuildDate:"2020-04-08T17:30:47Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
- DANM configuration (K8s manifests, kubeconfig files, CNI config file):
cat /etc/cni/net.d/00-danm.conf
{
"cniVersion": "0.3.1",
"name": "danm_meta_cni",
"type": "danm",
"kubeconfig": "/etc/cni/net.d/danm-kubeconfig",
"cniDir": "/etc/cni/net.d",
"namingScheme": "legacy"
}
cat /etc/cni/net.d/danm-kubeconfig
---
apiVersion: v1
kind: Config
current-context: default
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM5akNDQWQ2Z0F3SUJBZ0lJbnA4c0grdHJsemt3RFFZSktvWklodmNOQVFFTEJRQXdJVEVmTUIwR0ExVUUKQXd3V1kyOXRiWE5qYjNCbFlteHlhWEJ6WldOaFpHMXBiakFlRncweU1UQTNNRE14TURVek5UZGFGdzAwTVRBMwpNRE14TURVek5UZGFNQ0V4SHpBZEJnTlZCQU1NRm1OdmJXMXpZMjl3WldKc2NtbHdjMlZqWVdSdGFXNHdnZ0VpCk1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLQW9JQkFRQ3NIU0FWMURSTUFnZ1FtTng1TU5vL0c5V3YKR3hqa2luN0NWOHpYdVV6TFZtRjdjMG5KdjlVaDR1WG1LdzNFR3V4MnQveGx3NlhTN3c3TjdFSzcxNzF5Wm1nVwpXc3RPMWgzeDBDWVhoQ1VxOXVXNUdET2lZVnlxMUljOUZSSUQ0U2N6Z1FNb1hFT0ZhVDhlY1U0MEJNNTJqYmlnCkphb2lDZ0RVYmtNT3dMOWpTdlgzQlhpNjNybVRiZTlRMkM0dmxMUmdQK1lFRnpPaDJEYTU4cm5EdVJqVFUyaWsKOUIvNnVzeXMzZ0VFZkhxZ3R4NGNTQ0RVaFpxNVRjYTZjM2hUYVV6NU9HV3NNN2c1bEFDVDlNQ0tOOEN3R1ZNTwpBVkt2WkhPaDRYTWZlU0x3RjZHdkNBN1V6QzhYYjM3TU04REtVbjVQYnJyVUwvNUdrb1djNzI1R3QrMVJBZ01CCkFBR2pNakF3TUE4R0ExVWRFd0VCL3dRRk1BTUJBZjh3SFFZRFZSME9CQllFRkc0ZGg3bTZWcXd1bnZlN1ZKV28KcXJ1dFRmeWFNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUNXNjlsNlFUMjkwUkl0cXJuRldvZ05SV3dYQTlKbgpWSndCK2JsUmhoLzd2ODlTazZvWFhJZlZob0xGaDlZSEtYcFJaREd3RkVBWERTbFhUWFM1YlB6T3F2dkVmR0dCCm81TGRnZE1PR2VYMVg0OHB2TjNlUmZrRE54MFFEKzRFeVhBaVFobmMva0VJcDV0TTNMS1VEOUp5b2lCek5QL0wKSndZUGxiMkxHWFlQY0ZrRWMwTzJ6ZkZtUEtGL294ZFE0dWtDQU8zbVlFT1BEZEl6ZEI4dEYvdFBqUzQ0aWdlegpSUkRtN1RpR0wwd1VYZHQ1NlB6aTJyVEpwa3dqVHlZWTBscWNWZnBYcWkxc0R0QXBzRTRQTlZZZTVkTFdHYjRnCnpTTDRJSGtQejRMNnZnQUxKTVlWclVzNFBzZTVjZW9Gdm9vbHV6YjQ3SXpHUFljN3FiU09lNGdaCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0=
server: https://10.96.0.1:443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: danm
name: default
users:
- name: danm
user:
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IklseWJIYlBYQlJjOEJ1dHZ0Z3lOd3BLVkp1c3VJUEd3bkVRYm9aUWpFS1UifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYW5tLXRva2VuLXE2ZjZoIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRhbm0iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIzN2YzYzg1ZS03ZDlkLTQ2NTgtOGU4YS0yOGJmNmMwNjgyYTgiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGFubSJ9.UH1RlL8H3a9as0fTCPQNVU84MwDqI7QbORV8hQyj7EawBB8xi3EZWHUzWaf2K6TaIXZebWc_9x2odZDIrPQucVuQqBoUmTsiI083bPwbFxpita4_N_sjdHC-VwWWyDw7KZpfSWhapLRAMTTRzb_s_VO1-VP9Pv7wGj6kCB50CsnYn-FDQf2d-uPEjjNJlhtlSU9ggwWiuceRylW_kk0n554uAJkexCooW8gfsfv2qQWOiAR9cw7oDSstBcvxDUMGW1iTkpy2b1xHP_LRrXfi6XiJRE91kk4OApB83XaPm04DNcnNnCq0BC8qKjBx1NJmOZyqKHw_qoVbxU1GHQM1jg
preferences: {}
- OS (e.g. from /etc/os-release):
cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
- Kernel (e.g.
uname -a
):
uname -a
Linux controller-0 3.10.0-1160.15.2.rt56.1152.el7.tis.4.x86_64 #1 SMP PREEMPT RT Wed Jun 9 20:40:45 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux
- Others:
Help required
Any updates regarding this? Want to know cause for the issue