Failed to watch *v1alpha1.CNINode
Omar-Bishtawi opened this issue · 2 comments
What happened:
E1126 14:10:15.375095 11 reflector.go:147] pkg/mod/k8s.io/client-go@v0.28.3/tools/cache/reflector.go:229: Failed to watch *v1alpha1.CNINode: unknown (get cninodes.vpcresources.k8s.aws)
the above issue happened to us after upgrading vpc-cni from v1.14.1-eksbuild.1 to v1.15.4-eksbuild.1 this issue happened due to missing watch action in the cluster role policy for CNINode resource.
the issue is resolved once we added watch verb to CNINode in the cluster role resource. but the watch verb isn't included in the default manifest provided for 1.15.4 release link
What you expected to happen:
for the watch verb to be included by default in vpc-cni cluster role manifest if it's needed, if it's not needed what is the correct way to resolve this issue?
How to reproduce it :
don't have exact reproducing steps, it happened to me while upgrading the plugin on a normal eks cluster. the only thing that might be related is that we have SecurityGroupPolicy from the same API group(vpcresources.k8s.aws) as CNINodes set in the cluster.
Anything else we need to know?:
we use eks addons to install and handle the upgrade of VPC-CNI
Environment:
- Kubernetes version (use
kubectl version):
Client Version: version.Info{Major:"1", Minor:"27+", GitVersion:"v1.27.1-eks-2f008fe", GitCommit:"abfec7d7e55d56346a5259c9379dea9f56ba2926", GitTreeState:"clean", BuildDate:"2023-04-14T20:43:13Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"darwin/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"28+", GitVersion:"v1.28.3-eks-4f4795d", GitCommit:"e77944ea667aee90c84c48a74a2a0da7a23e0508", GitTreeState:"clean", BuildDate:"2023-10-20T23:21:39Z", GoVersion:"go1.20.10", Compiler:"gc", Platform:"linux/amd64"}
- CNI Version: v1.15.4
- OS (e.g:
cat /etc/os-release): Bottlerocket - Kernel (e.g.
uname -a): arm64
Duplicate of #2589
@Omar-Bishtawi this error log is harmless. The IPAM daemon does not need to WATCH the CNINode resource, as it has no need to cache it. Not passing the schema is supposed to prevent the cache from issuing a WATCH (https://github.com/aws/amazon-vpc-cni-k8s/blob/master/pkg/k8sapi/k8sutils.go#L113)... but that does not appear to be working.
Adding the watch permission is fine, as this resource is small enough that caching it does not add any noticeable memory overhead. I will add the watch permission in a new PR
⚠️ COMMENT VISIBILITY WARNING⚠️
Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.