Community operator pod is crashing
cmoulliard opened this issue · 2 comments
cmoulliard commented
Issue
The community operator pod is crashing and reports the following error invalid source, secret specified is malformed - upstream-community-operators
on okd 3.11
time="2019-04-04T07:10:02Z" level=info msg="Using in-cluster kube client config" port=50051 type=appregistry
--
| time="2019-04-04T07:10:02Z" level=info msg="operator source(s) specified are - [https://quay.io/cnr\|community-operators --registry=https://quay.io/cnr\|upstream-community-operators ]" port=50051 type=appregistry
| time="2019-04-04T07:10:02Z" level=info msg="package(s) specified are - percona,postgresql,strimzi-kafka-operator,node-problem-detector,automationbroker,kiecloud-operator,elasticsearch-operator,node-network-operator,microcks,prometheus,jaeger,cluster-logging,planetscale,cockroachdb,metering,descheduler,oneagent,templateservicebroker,federation,etcd,camel-k" port=50051 type=appregistry
| time="2019-04-04T07:10:02Z" level=info msg="can't proceed, bailing out" port=50051 type=appregistry
| time="2019-04-04T07:10:02Z" level=error msg="the following error(s) occurred while parsing input - invalid source, secret specified is malformed - upstream-community-operators" port=50051 type=appregistry
| time="2019-04-04T07:10:02Z" level=fatal msg="error loading manifest from remote registry - invalid source, secret specified is malformed - upstream-community-operators" port=50051 type=appregistry
Pod deployed
apiVersion: v1
kind: Pod
metadata:
annotations:
openshift.io/scc: restricted
creationTimestamp: '2019-04-04T06:53:28Z'
generateName: community-operators-664fc79df5-
labels:
marketplace.catalogSourceConfig: community-operators
pod-template-hash: '2209735891'
name: community-operators-664fc79df5-j8n92
namespace: marketplace
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: ReplicaSet
name: community-operators-664fc79df5
uid: 5a36e2c0-56a6-11e9-b581-080027f60d62
resourceVersion: '31663'
selfLink: /api/v1/namespaces/marketplace/pods/community-operators-664fc79df5-j8n92
uid: 5a3c5ef5-56a6-11e9-b581-080027f60d62
spec:
containers:
- command:
- appregistry-server
- >-
--registry=https://quay.io/cnr|community-operators
--registry=https://quay.io/cnr|upstream-community-operators
- '-o'
- >-
percona,postgresql,strimzi-kafka-operator,node-problem-detector,automationbroker,kiecloud-operator,elasticsearch-operator,node-network-operator,microcks,prometheus,jaeger,cluster-logging,planetscale,cockroachdb,metering,descheduler,oneagent,templateservicebroker,federation,etcd,camel-k
image: quay.io/openshift/origin-operator-registry
imagePullPolicy: Always
livenessProbe:
exec:
command:
- grpc_health_probe
- '-addr=localhost:50051'
failureThreshold: 30
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: community-operators
ports:
- containerPort: 50051
name: grpc
protocol: TCP
readinessProbe:
exec:
command:
- grpc_health_probe
- '-addr=localhost:50051'
failureThreshold: 30
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
securityContext:
capabilities:
drop:
- KILL
- MKNOD
- SETGID
- SETUID
runAsUser: 1000200000
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-pgrsw
readOnly: true
dnsPolicy: ClusterFirst
imagePullSecrets:
- name: default-dockercfg-kkjkn
nodeName: localhost
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 1000200000
seLinuxOptions:
level: 's0:c14,c9'
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
volumes:
- name: default-token-pgrsw
secret:
defaultMode: 420
secretName: default-token-pgrsw
status:
conditions:
- lastProbeTime: null
lastTransitionTime: '2019-04-04T06:53:29Z'
status: 'True'
type: Initialized
- lastProbeTime: null
lastTransitionTime: '2019-04-04T06:53:29Z'
message: 'containers with unready status: [community-operators]'
reason: ContainersNotReady
status: 'False'
type: Ready
- lastProbeTime: null
lastTransitionTime: null
message: 'containers with unready status: [community-operators]'
reason: ContainersNotReady
status: 'False'
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: '2019-04-04T06:53:29Z'
status: 'True'
type: PodScheduled
containerStatuses:
- containerID: >-
docker://26d42781129f924cf8586848d3dff3665056ef16f0198c3df6c53f9db7f36b7a
image: 'quay.io/openshift/origin-operator-registry:latest'
imageID: >-
docker-pullable://quay.io/openshift/origin-operator-registry@sha256:aa424fce55ad5a28b19fcc9399cba6d62bc04a6d9a4730dfd150e9b36d4262f6
lastState:
terminated:
containerID: >-
docker://26d42781129f924cf8586848d3dff3665056ef16f0198c3df6c53f9db7f36b7a
exitCode: 1
finishedAt: '2019-04-04T07:15:18Z'
message: >
time="2019-04-04T07:15:18Z" level=fatal msg="error loading manifest
from remote registry - invalid source, secret specified is malformed
- upstream-community-operators" port=50051 type=appregistry
reason: Error
startedAt: '2019-04-04T07:15:18Z'
name: community-operators
ready: false
restartCount: 9
state:
waiting:
message: >-
Back-off 5m0s restarting failed container=community-operators
pod=community-operators-664fc79df5-j8n92_marketplace(5a3c5ef5-56a6-11e9-b581-080027f60d62)
reason: CrashLoopBackOff
hostIP: 10.0.3.15
phase: Running
podIP: 172.17.0.13
qosClass: BestEffort
startTime: '2019-04-04T06:53:29Z'
aravindhp commented
Please see #154 (comment)
cmoulliard commented
I did a new test and followed your suggestion and that works
time="2019-04-04T14:50:35Z" level=info msg="Using in-cluster kube client config" port=50051 type=appregistry
time="2019-04-04T14:50:35Z" level=info msg="operator source(s) specified are - [https://quay.io/cnr|upstream-community-operators ]" port=50051 type=appregistry
time="2019-04-04T14:50:35Z" level=info msg="package(s) specified are - federation,etcd,couchbase-enterprise,mongodb-enterprise,planetscale,redis-enterprise,strimzi-kafka-operator,microcks,percona,oneagent,cockroachdb,jaeger,aws-service,prometheus,postgresql,vault" port=50051 type=appregistry
time="2019-04-04T14:50:35Z" level=info msg="input has been sanitized" port=50051 type=appregistry
time="2019-04-04T14:50:35Z" level=info msg="sources: [https://quay.io/cnr/upstream-community-operators ]" port=50051 type=appregistry
time="2019-04-04T14:50:35Z" level=info msg="packages: [federation etcd couchbase-enterprise mongodb-enterprise planetscale redis-enterprise strimzi-kafka-operator microcks percona oneagent cockroachdb jaeger aws-service prometheus postgresql vault]" port=50051 type=appregistry
time="2019-04-04T14:50:36Z" level=info msg="resolved the following packages: [upstream-community-operators/postgresql:0.0.2 upstream-community-operators/federation:0.0.2 upstream-community-operators/vault:0.0.1 upstream-community-operators/jaeger:0.0.2 upstream-community-operators/aws-service:0.0.3 upstream-community-operators/redis-enterprise:0.0.2 upstream-community-operators/strimzi-kafka-operator:0.0.3 upstream-community-operators/etcd:0.0.5 upstream-community-operators/prometheus:0.0.1 upstream-community-operators/couchbase-enterprise:0.0.5 upstream-community-operators/cockroachdb:0.0.1 upstream-community-operators/oneagent:0.0.1 upstream-community-operators/microcks:0.0.1 upstream-community-operators/percona:0.0.4 upstream-community-operators/mongodb-enterprise:0.0.2 upstream-community-operators/planetscale:0.0.2]" port=50051 type=appregistry
time="2019-04-04T14:50:36Z" level=info msg="downloading repository: upstream-community-operators/postgresql:0.0.2 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:39Z" level=info msg="downloading repository: upstream-community-operators/federation:0.0.2 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:40Z" level=info msg="downloading repository: upstream-community-operators/vault:0.0.1 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:42Z" level=info msg="downloading repository: upstream-community-operators/jaeger:0.0.2 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:45Z" level=info msg="downloading repository: upstream-community-operators/aws-service:0.0.3 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:46Z" level=info msg="downloading repository: upstream-community-operators/redis-enterprise:0.0.2 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:48Z" level=info msg="downloading repository: upstream-community-operators/strimzi-kafka-operator:0.0.3 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:50Z" level=info msg="downloading repository: upstream-community-operators/etcd:0.0.5 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:53Z" level=info msg="downloading repository: upstream-community-operators/prometheus:0.0.1 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:54Z" level=info msg="downloading repository: upstream-community-operators/couchbase-enterprise:0.0.5 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:56Z" level=info msg="downloading repository: upstream-community-operators/cockroachdb:0.0.1 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:50:59Z" level=info msg="downloading repository: upstream-community-operators/oneagent:0.0.1 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:51:01Z" level=info msg="downloading repository: upstream-community-operators/microcks:0.0.1 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:51:03Z" level=info msg="downloading repository: upstream-community-operators/percona:0.0.4 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:51:05Z" level=info msg="downloading repository: upstream-community-operators/mongodb-enterprise:0.0.2 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:51:07Z" level=info msg="downloading repository: upstream-community-operators/planetscale:0.0.2 from https://quay.io/cnr " port=50051 type=appregistry
time="2019-04-04T14:51:09Z" level=info msg="download complete - 16 repositories have been downloaded" port=50051 type=appregistry
time="2019-04-04T14:51:10Z" level=info msg="all manifest(s) have been merged into one" port=50051 type=appregistry
time="2019-04-04T14:51:10Z" level=info msg="loading into sqlite database" port=50051 type=appregistry
time="2019-04-04T14:51:10Z" level=info msg="using configmap loader to build sqlite database" port=50051 type=appregistry
time="2019-04-04T14:51:10Z" level=info msg="loading CRDs" port=50051 type=appregistry
time="2019-04-04T14:51:10Z" level=info msg="loading Bundles" port=50051 type=appregistry
time="2019-04-04T14:51:10Z" level=info msg="loading Packages" port=50051 type=appregistry
time="2019-04-04T14:51:10Z" level=info msg="extracting provided API information" port=50051 type=appregistry
time="2019-04-04T14:51:10Z" level=info msg="serving registry" port=50051 type=appregistry