Unable to start pod due to wrong CRD version
Closed this issue · 12 comments
Description
[Description of the problem]
Steps to Reproduce
[Brief description of the steps you took to encounter the problem, if applicable]
Expected behavior: [What you expected to happen]
Actual behavior: [What actually happened]
Versions
[Which version of STUNner you are using]
Info
[Please copy-paste the output of the below commands and make sure to remove all sensitive information, like usernames, passwords, IP addresses, etc.]
Gateway API status
[Output of kubectl get gateways,gatewayconfigs,gatewayclasses,udproutes.stunner.l7mp.io --all-namespaces -o yaml]
Operator logs
[Output of kubectl -n stunner-system logs $(kubectl get pods -l control-plane=stunner-gateway-operator-controller-manager --all-namespaces -o jsonpath='{.items[0].metadata.name}')]
024-10-22T11:25:56.835450039Z INFO setup starting operator thread
2024-10-22T11:25:56.835501009Z INFO gatewayconfig-controller Created GatewayConfig controller
2024-10-22T11:25:56.835512302Z INFO gatewayconfig-controller Watching GatewayConfig objects
2024-10-22T11:25:56.846979851Z INFO gatewayconfig-controller Watching Secret objects
2024-10-22T11:25:56.847008073Z INFO dataplane-controller created dataplane controller
2024-10-22T11:25:56.847013364Z INFO dataplane-controller watching dataplane objects
2024-10-22T11:25:56.847019738Z INFO gateway-controller Created Gateway controller
2024-10-22T11:25:56.847025445Z INFO gateway-controller Watching GatewayClass objects
2024-10-22T11:25:56.847029244Z INFO gateway-controller Watching Gateway objects
2024-10-22T11:25:56.856714629Z ERROR setup problem running operator {"error": "Cannot register gateway controller: no matches for kind "Gateway" in version "gateway.networking.k8s.io/v1""}
hi @hudson-s !
This error is the key:
{"error": "Cannot register gateway controller: no matches for kind "Gateway" in version "gateway.networking.k8s.io/v1""}
I guess Gateway API v1 CRD is somehow not installed in your cluster (install help: https://gateway-api.sigs.k8s.io/guides/#install-standard-channel).
Is this a new STUNner install or you just updated from an earlier version? BTW mind to tell a few words about your use-case?
hi @hudson-s !
This error is the key:
{"error": "Cannot register gateway controller: no matches for kind "Gateway" in version "gateway.networking.k8s.io/v1""}I guess Gateway API v1 CRD is somehow not installed in your cluster (install help: https://gateway-api.sigs.k8s.io/guides/#install-standard-channel).
Is this a new STUNner install or you just updated from an earlier version? BTW mind to tell a few words about your use-case?
I am using the helm source file for a new installation
- cd /stunner-helm-main/helm/stunner-gateway-operator
- helm install stunner-gateway-operator . --create-namespace --namespace=stunner-system
In addition:
kubectl api-resources | grep gateway
gatewayclasses gc gateway.networking.k8s.io/v1beta1 false GatewayClass
gateways gtw gateway.networking.k8s.io/v1beta1 true Gateway
grpcroutes gateway.networking.k8s.io/v1alpha2 true GRPCRoute
httproutes gateway.networking.k8s.io/v1beta1 true HTTPRoute
referencegrants refgrant gateway.networking.k8s.io/v1beta1 true ReferenceGrant
tcproutes gateway.networking.k8s.io/v1alpha2 true TCPRoute
tlsroutes gateway.networking.k8s.io/v1alpha2 true TLSRoute
Which STUNner version is it? We upgraded the Gateway API CRD in our helm chart to v1.0.0 10 month ago: https://github.com/l7mp/stunner-helm/blob/main/helm/stunner-gateway-operator/crds/gateway-api-crd.yaml
Can you give info on your cluster? Is it self-managed or do you use managed K8s (e.g., GKE, AKS, EKS, etc.)?
Which STUNner version is it? We upgraded the Gateway API CRD in our helm chart to v1.0.0 10 month ago: https://github.com/l7mp/stunner-helm/blob/main/helm/stunner-gateway-operator/crds/gateway-api-crd.yaml
Can you give info on your cluster? Is it self-managed or do you use managed K8s (e.g., GKE, AKS, EKS, etc.)?
I am use managed by alibaba cloud k8s.
I am using the latest clone
Interesting, can you double check that the CRD install was successful?
To install the CRDS, Just run kubectl apply -f helm/stunner-gateway-operator/crds/ and check the output, and the available API resources and CRDs in your cluster. I have a new insall on AKS, and the output is the following:
kubectl api-resources | grep gateway
gatewayclasses gc gateway.networking.k8s.io/v1 false GatewayClass
gateways gtw gateway.networking.k8s.io/v1 true Gateway
grpcroutes gateway.networking.k8s.io/v1alpha2 true GRPCRoute
httproutes gateway.networking.k8s.io/v1 true HTTPRoute
referencegrants refgrant gateway.networking.k8s.io/v1beta1 true ReferenceGrant
tcproutes gateway.networking.k8s.io/v1alpha2 true TCPRoute
tlsroutes gateway.networking.k8s.io/v1alpha2 true TLSRoute
udproutes gateway.networking.k8s.io/v1alpha2 true UDPRoute
gatewayconfigs gwconf stunner.l7mp.io/v1 true GatewayConfig
Note that we also see gatewayconfigs.stunner.l7mp.io/v1 resources in the output.
kubectl get crds | grep gateway
gatewayclasses.gateway.networking.k8s.io 2024-10-21T17:55:51Z
gatewayconfigs.stunner.l7mp.io 2024-10-21T17:55:51Z
gateways.gateway.networking.k8s.io 2024-10-21T17:55:51Z
grpcroutes.gateway.networking.k8s.io 2024-10-21T17:55:51Z
httproutes.gateway.networking.k8s.io 2024-10-21T17:55:51Z
referencegrants.gateway.networking.k8s.io 2024-10-21T17:55:51Z
tcproutes.gateway.networking.k8s.io 2024-10-21T17:55:51Z
tlsroutes.gateway.networking.k8s.io 2024-10-21T17:55:51Z
udproutes.gateway.networking.k8s.io 2024-10-21T17:55:51Z
Interesting, can you double check that the CRD install was successful?
To install the CRDS, Just run
kubectl apply -f helm/stunner-gateway-operator/crds/and check the output, and the available API resources and CRDs in your cluster. I have a new insall on AKS, and the output is the following:kubectl api-resources | grep gateway gatewayclasses gc gateway.networking.k8s.io/v1 false GatewayClass gateways gtw gateway.networking.k8s.io/v1 true Gateway grpcroutes gateway.networking.k8s.io/v1alpha2 true GRPCRoute httproutes gateway.networking.k8s.io/v1 true HTTPRoute referencegrants refgrant gateway.networking.k8s.io/v1beta1 true ReferenceGrant tcproutes gateway.networking.k8s.io/v1alpha2 true TCPRoute tlsroutes gateway.networking.k8s.io/v1alpha2 true TLSRoute udproutes gateway.networking.k8s.io/v1alpha2 true UDPRoute gatewayconfigs gwconf stunner.l7mp.io/v1 true GatewayConfigNote that we also see gatewayconfigs.stunner.l7mp.io/v1 resources in the output.
kubectl get crds | grep gateway gatewayclasses.gateway.networking.k8s.io 2024-10-21T17:55:51Z gatewayconfigs.stunner.l7mp.io 2024-10-21T17:55:51Z gateways.gateway.networking.k8s.io 2024-10-21T17:55:51Z grpcroutes.gateway.networking.k8s.io 2024-10-21T17:55:51Z httproutes.gateway.networking.k8s.io 2024-10-21T17:55:51Z referencegrants.gateway.networking.k8s.io 2024-10-21T17:55:51Z tcproutes.gateway.networking.k8s.io 2024-10-21T17:55:51Z tlsroutes.gateway.networking.k8s.io 2024-10-21T17:55:51Z udproutes.gateway.networking.k8s.io 2024-10-21T17:55:51Z
I just ran the kubectl apply -f helm/stunner-gateway-operator/crds/
Print as follows
# kubectl api-resources | grep gateway
gatewayclasses gc gateway.networking.k8s.io/v1 false GatewayClass
gateways gtw gateway.networking.k8s.io/v1beta1 true Gateway
grpcroutes gateway.networking.k8s.io/v1alpha2 true GRPCRoute
httproutes gateway.networking.k8s.io/v1 true HTTPRoute
referencegrants refgrant gateway.networking.k8s.io/v1beta1 true ReferenceGrant
tcproutes gateway.networking.k8s.io/v1alpha2 true TCPRoute
tlsroutes gateway.networking.k8s.io/v1alpha2 true TLSRoute
udproutes gateway.networking.k8s.io/v1alpha2 true UDPRoute
gatewayconfigs gwconf stunner.l7mp.io/v1 true GatewayConfig
# kubectl get crds | grep gateway
gatewayclasses.gateway.networking.k8s.io 2024-08-09T03:29:06Z
gatewayconfigs.stunner.l7mp.io 2024-10-23T12:15:16Z
gateways.gateway.networking.k8s.io 2024-08-09T03:29:06Z
grpcroutes.gateway.networking.k8s.io 2024-10-23T12:15:16Z
httproutes.gateway.networking.k8s.io 2024-08-09T03:29:06Z
referencegrants.gateway.networking.k8s.io 2024-08-09T03:29:06Z
tcproutes.gateway.networking.k8s.io 2024-10-23T12:15:16Z
tlsroutes.gateway.networking.k8s.io 2024-10-23T12:15:16Z
udproutes.gateway.networking.k8s.io 2024-10-23T12:15:16Z
I am not very proficient in crd. Can you tell me what I need to do next
LGTM. Any update on your deployment? Does it finally work?
LGTM. Any update on your deployment? Does it finally work?
sorry ,no
I need your help
hello I tried again and resolved the matter. But there is a line printed in the pod, I don't know if it has any effect
2024-10-24T02:34:01.582269558Z INFO renderer No gateway-class objects found {"event": "render: generation: 1"}
2024-10-24T02:34:01.788381301Z INFO udproute-controller Validating backend {"udproute": "not found"}
2024-10-24T02:34:01.789473895Z INFO udproute-controller Validating backend {"udproute": "not found"}
2024-10-24T02:34:12.852483088Z INFO udproute-controller Validating backend {"udproute": "not found"}
2024-10-24T02:34:12.852514074Z INFO udproute-controller Validating backend {"udproute": "not found"}
2024-10-24T02:35:45.4385956Z INFO node-controller failed to find node with valid external address {"reason": "End of node list reached after searching through 4 node(s)"}
2024-10-24T02:35:45.688628843Z INFO operator Starting new reconcile generation {"generation": 1, "last-acked-generation": -1}
2024-10-24T02:35:45.68866041Z INFO renderer Rendering configuration {"generation": 2, "event": "render: generation: 2"}
2024-10-24T02:35:45.688665757Z INFO renderer Starting dataplane render {"mode": "managed"}
2024-10-24T02:35:45.688678733Z INFO renderer No gateway-class objects found {"event": "render: generation: 2"}
Q1:
udproute-controller Validating backend {"udproute": "not found"}
Q2
failed to find node with valid external address {"reason": "End of node list reached after searching through 4 node(s)"}
I understand because I didn't leak the extranet ip node.
- If the Internet ip address is leaked, will the entire cluster have security problems?
- Do you need taint?
I would recommend to try a working example first. The simple-tunnel shows how to configure STUNner. Let us know whether this example works in your setup or not.
Closing this as a preparation for rolling the v1 release, please reopen if you have further info.