Operator gets stuck at syncing the statefull set when container resources are specified in a non "normal" form
stocksr opened this issue · 3 comments
With this in my crd
redis:
image: redis:6.2.6-alpine
initialDelaySeconds: 10
resources:
limits:
cpu: 1000m
memory: 2Gi
requests:
cpu: 500m
memory: 0.8Gi
the stateful set gets created with
resources:
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: 500m
memory: 858993459200m
and the operator keeps logging
... "logger":"controller_redis","msg":"Applied *v1.StatefulSet","Namespace ...
every 2 seconds
if I change my definition to
limits:
cpu: "1"
memory: 2Gi
requests:
cpu: 500m
memory: 800Mi
the stateful set is created with identical values and the operator moves on to the next step.
P.S. To help other people find this issue: everything was "running" but there was no master label assigned to any of the pods, so I was unable to connect and the pod logs all showed the instances running as standalone servers, the ConfigMap did not have the replicaof line.
I guess something is normalising the resource values as the stateful set is created, but not normalising them before the diff check.
Literally experienced the same issue today because we changed some resources fields.
What happened was that the new pods had no labels "role=master"/"role=replica", and the ConfigMap contained no "REPLICA OF" directive.
@stocksr we found that our issue was related to using float numbers, meaning that we have been specifying limits and requests with e.g. "0.3". As soon as we switched to "300m" the issue went away.
Whenever we use any floats, we can see labels not appearing in the pods.
superseded by nrvnrvn/k8dis#10