banzaicloud/koperator

Support for setting up koperator

login21 opened this issue · 6 comments

login21 commented

Trying to install koperator on my openshift cluster and followed the steps mentioned on page but not able to complete my setup.

Steps I followed :

  1. Ran the install.sh script on my oc cluster
    curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.20.0/install.sh | bash -s v0.20.0

  2. Ran the command to install the operator in my cluster

command : $ oc create -f https://operatorhub.io/install/banzaicloud-kafka-operator.yaml
output : subscription.operators.coreos.com/my-banzaicloud-kafka-operator created

  1. when trying to see resources for my koperator I don't see any

command: $ oc get csv -n operators
output:
NAME DISPLAY VERSION REPLACES PHASE
nginx-ingress-operator.v0.5.0 Nginx Ingress Operator 0.5.0 nginx-ingress-operator.v0.4.0 Installing

It only shows the nginx operator which I installed earlier.

Can anyone please help me figure out why my operator is not coming up as I don't see any errors while doing the setup so I can debug further

Thanks!

login21 commented

Thank you so much @stoader for your response.

I followed the link which you have shared and my cert-manager, zookeeper and prometheus pods are running fine. When I install koperator helm chart my koperator pods are going in CrashloopBackOff error state. When I see the log for pod I get this error.

This is what I see when I run log command

oc logs pod/kafka-operator-operator-654c4d8c56-z2w6h -p -n kafka
error: a container name must be specified for pod kafka-operator-operator-654c4d8c56-z2w6h, choose one of: [kube-rbac-proxy manager]

and output from the pod describe command.

Containers:
kube-rbac-proxy:
Container ID: cri-o://84c5a83327770ab5fa2d24b348fbba8c2fa158c1c9ea085b43e7f5f17ba51baf
Image: gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0
Image ID: gcr.io/kubebuilder/kube-rbac-proxy@sha256:34e8724e0f47e31eb2ec3279ac398b657db5f60f167426ee73138e2e84af6486
Port: 8443/TCP
Host Port: 0/TCP
Args:
--secure-listen-address=0.0.0.0:8443
--upstream=http://127.0.0.1:8080/
--logtostderr=true
--v=10
State: Running
Started: Fri, 11 Feb 2022 05:53:34 +0000
Ready: True
Restart Count: 0
Environment:
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nx6ft (ro)
manager:
Container ID: cri-o://21e680e4c73cb6ecda0c6c00fdf8dfcf1643eba886023bd5177eeeecd0251474
Image: ghcr.io/banzaicloud/kafka-operator:v0.20.2
Image ID: ghcr.io/banzaicloud/kafka-operator@sha256:e341aefa9a908b2edceaaec910c8e17a0481d5bb0c7d9f8102ddf3643a9d2141
Ports: 443/TCP, 8080/TCP, 9001/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
Command:
/manager
Args:
--enable-leader-election
--cert-manager-namespace=cert-manager
--cert-manager-enabled=false
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Fri, 11 Feb 2022 06:05:10 +0000
Finished: Fri, 11 Feb 2022 06:05:17 +0000
Ready: False
Restart Count: 7
Limits:
cpu: 200m
memory: 256Mi
Requests:
cpu: 100m
memory: 128Mi
Environment:
POD_NAMESPACE: kafka (v1:metadata.namespace)
Mounts:
/etc/webhook/certs from serving-cert (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nx6ft (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
serving-cert:
Type: Secret (a volume populated by a Secret)
SecretName: kafka-operator-serving-cert
Optional: false
kube-api-access-nx6ft:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional:
DownwardAPI: true
ConfigMapName: openshift-service-ca.crt
ConfigMapOptional:
QoS Class: Burstable
Node-Selectors:
Tolerations: node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 12m default-scheduler Successfully assigned kafka/kafka-operator-operator-654c4d8c56-z2w6h to gcp-gen-openshift-fp84t-worker-b-jlr4r.c.ngca-poc.internal
Normal AddedInterface 12m multus Add eth0 [10.129.2.247/23] from openshift-sdn
Normal Pulling 12m kubelet Pulling image "gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0"
Normal Pulled 12m kubelet Successfully pulled image "gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0" in 534.628328ms
Normal Created 12m kubelet Created container kube-rbac-proxy
Normal Started 12m kubelet Started container kube-rbac-proxy
Normal Pulling 12m kubelet Pulling image "ghcr.io/banzaicloud/kafka-operator:v0.20.2"
Normal Pulled 12m kubelet Successfully pulled image "ghcr.io/banzaicloud/kafka-operator:v0.20.2" in 3.647231638s
Normal Created 11m (x4 over 12m) kubelet Created container manager
Normal Started 11m (x4 over 12m) kubelet Started container manager
Normal Pulled 10m (x4 over 12m) kubelet Container image "ghcr.io/banzaicloud/kafka-operator:v0.20.2" already present on machine
Warning BackOff 2m10s (x43 over 12m) kubelet Back-off restarting failed container

Will you please help me figure out what can be the reason for this error. I was trying to find out resolution for this error og google and some places it saying the docker container and service might be mapped to different port and can cause this error. I am not sure how I can find this out with the helm chart. Please help with this.

stoader commented

Please attach koperator logs:
kubectl logs -n kafka deployment/kafka-operator-operator -c manager

login21 commented

Thank @stoader for your response again.
Please find the required logs here

Just to clarify, I am running this in openshift cluster
command : oc logs -n kafka deployment/kafka-operator-operator -c manager

output:
I0215 07:16:22.996344 1 request.go:665] Waited for 1.003860333s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/external.metrics.k8s.io/v1beta1?timeout=32s
{"level":"error","ts":"2022-02-15T07:16:28.899Z","logger":"setup","msg":"unable to start manager","error":"could not setup indexer for field: no matches for kind "KafkaTopic" in version "kafka.banzaicloud.io/v1alpha1"","errorVerbose":"no matches for kind "KafkaTopic" in version "kafka.banzaicloud.io/v1alpha1"\ncould not setup indexer for field\ngithub.com/banzaicloud/koperator/pkg/k8sutil.AddKafkaTopicIndexers.func1\n\t/workspace/pkg/k8sutil/cache.go:40\nsigs.k8s.io/controller-runtime/pkg/cluster.New\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/cluster/cluster.go:165\nsigs.k8s.io/controller-runtime/pkg/manager.New\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.3/pkg/manager/manager.go:312\nmain.main\n\t/workspace/main.go:127\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:255\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1581"}

stoader commented

Apparently you missed the first step from Install Koperator with Helm and you don't have the koperator related CRDs deployed in your cluster.

See the error message no matches for kind "KafkaTopic" in version "kafka.banzaicloud.io/v1alpha1" is complaining about the missing KafkaTopic.

stoader commented

@login21 note you can join the koperator's community slack channel were you can reach others who use koperator as well: https://banzaicloud.com/invite-slack/