Using Experiment driver moves prometheus pod to different nodes
sk1tter opened this issue · 2 comments
sk1tter commented
Version & Branch
main branch
Expected Behaviour
prometheus pod should be fixed to master node
Actual Behaviour
Experiment driver executes make clean which redeploys prometheus pods, which schedules prometheus on different nodes.
Steps to Reproduce the Problem
Additional Info
sk1tter commented
Also the master node having the taint key1=value1:NoSchedule causes promestheus to be scheduled on worker nodes. It would be better to add a toleration to pods that need to be scheduled on the master node.
sk1tter commented
knative-webhook pod fails to autoscale due to taint
│ Events: │
│ Type Reason Age From Message │
│ ---- ------ ---- ---- ------- │
│ Warning FailedScheduling 2m30s default-scheduler 0/21 nodes are available: 1 node(s) had taint {key1: value1}, that the pod didn't tolerate, 20 node(s) didn't match Pod's node affinity/selector. │
│ Warning FailedScheduling 14s (x1 over 74s) default-scheduler 0/21 nodes are available: 1 node(s) had taint {key1: value1}, that the pod didn't tolerate, 20 node(s) didn't match Pod's node affinity/selector.