EKS EBS CSI addon - New node group issue "add cmd: failed to assign an IP address to container"
MarcoPolaris opened this issue · 3 comments
What happened:
I'm facing issue when trying to create a new node group to my EKS Cluster.
Environment:
- Kubernetes version (use
kubectl version): 1.28 - CNI Version: v1.30.0-eksbuild.1
Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3eb650e5b06f909321c170c75369c9626b234b410abd784c45c685197689fa88": plugin type="aws-cni" name="aws-cni" failed (add): add cmd: failed to assign an IP address to container
These are available IP in my 3 subnets cluster.
This is ipamd.log output:
tail ipamd.log
{"level":"debug","ts":"2024-05-22T14:43:38.048Z","caller":"ipamd/ipamd.go:815","msg":"Node found \"ip-10-0-2-229.eu-south-1.compute.internal\" - no of taints - 1"}
{"level":"debug","ts":"2024-05-22T14:43:38.048Z","caller":"ipamd/ipamd.go:683","msg":"Recently we had InsufficientCidr error hence will wait for 2m0s before retrying"}
{"level":"debug","ts":"2024-05-22T14:43:38.048Z","caller":"ipamd/ipamd.go:1303","msg":"ENI eni-010742ce24f96fb7d cannot be deleted because it is primary"}
{"level":"debug","ts":"2024-05-22T14:43:43.050Z","caller":"ipamd/ipamd.go:682","msg":"IP pool is too low: available (0) < ENI target (1) * addrsPerENI (16)"}
{"level":"debug","ts":"2024-05-22T14:43:43.050Z","caller":"ipamd/ipamd.go:2163","msg":"Prefix pool stats: Total IPs/Prefixes = 0/0, AssignedIPs/CooldownIPs: 0/0, c.maxIPsPerENI = 176"}
{"level":"debug","ts":"2024-05-22T14:43:43.050Z","caller":"ipamd/ipamd.go:683","msg":"Starting to increase pool size"}
{"level":"debug","ts":"2024-05-22T14:43:43.050Z","caller":"ipamd/ipamd.go:803","msg":"Prefix target is 1, short of 1 prefixes, free 0 prefixes"}
{"level":"debug","ts":"2024-05-22T14:43:43.050Z","caller":"ipamd/ipamd.go:815","msg":"Node found \"ip-10-0-2-229.eu-south-1.compute.internal\" - no of taints - 1"}
{"level":"debug","ts":"2024-05-22T14:43:43.050Z","caller":"ipamd/ipamd.go:683","msg":"Recently we had InsufficientCidr error hence will wait for 2m0s before retrying"}
{"level":"debug","ts":"2024-05-22T14:43:43.050Z","caller":"ipamd/ipamd.go:1303","msg":"ENI eni-010742ce24f96fb7d cannot be deleted because it is primary"}
Any help/suggestion on how to solve this? I can't understand why ip pool available is 0
It is not about the number of IP Addresses, but the number of free ip addresses in your Subnet CIDR. Can you inspect your Subnet CIDRs and see if ip allocation is equally distributed and try to launch your node group in the subnet (with CIDR) having sufficient number of free ip addresses?
Closing due to lack of further information.
This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.