autoscaler scales to minimum when jenkins controller is unavailable
shamil opened this issue · 0 comments
shamil commented
When jenkins controller is unavailable, autoscaler thinks that that there is 0 agents running and adjust the number of nodes to one.
The proper operation should be if jenkins controller is inaccessible, all scaling activities should be suspended until the it can contact the controller.
Here are the relevant logs:
time="2022-07-10T19:45:18+03:00" level=error msg="can't get current jenkins usage: Get \"http://jenkins-master.service.integration.consul:8080/computer/api/json?depth=1\": dial tcp: lookup jenkins-master.service.integration.consul on 10.11.240.10:53: no such host" backend=gce component=scaler dryRun=false
time="2022-07-10T19:46:18+03:00" level=debug msg="current nodes usage is -9223372036854775808%" backend=gce component=scaler dryRun=false
time="2022-07-10T19:46:18+03:00" level=debug msg="number of nodes: 0" backend=gce component=scaler dryRun=false
time="2022-07-10T19:46:18+03:00" level=debug msg="is working hours now: false - cron: * 5-17 * * 0-5" backend=gce component=scaler dryRun=false
time="2022-07-10T19:46:18+03:00" level=debug msg="at or under minimum nodes. will adjust to the minimum if needed" backend=gce component=scaler dryRun=false
time="2022-07-10T19:46:18+03:00" level=info msg="not a single node off work hours. will adjust to one" backend=gce component=scaler dryRun=false
time="2022-07-10T19:47:18+03:00" level=debug msg="current nodes usage is 0%" backend=gce component=scaler dryRun=false
time="2022-07-10T19:47:18+03:00" level=debug msg="number of nodes: 1" backend=gce component=scaler dryRun=false
time="2022-07-10T19:47:18+03:00" level=debug msg="is working hours now: false - cron: * 5-17 * * 0-5" backend=gce component=scaler dryRun=false
time="2022-07-10T19:47:18+03:00" level=debug msg="at or under minimum nodes. will adjust to the minimum if needed" backend=gce component=scaler dryRun=false
time="2022-07-10T19:48:18+03:00" level=debug msg="current nodes usage is 0%" backend=gce component=scaler dryRun=false
time="2022-07-10T19:48:18+03:00" level=debug msg="number of nodes: 1" backend=gce component=scaler dryRun=false
time="2022-07-10T19:48:18+03:00" level=debug msg="is working hours now: false - cron: * 5-17 * * 0-5" backend=gce component=scaler dryRun=false
time="2022-07-10T19:48:18+03:00" level=debug msg="at or under minimum nodes. will adjust to the minimum if needed" backend=gce component=scaler dryRun=false
time="2022-07-10T19:49:18+03:00" level=debug msg="current nodes usage is 0%" backend=gce component=scaler dryRun=false
time="2022-07-10T19:49:18+03:00" level=debug msg="number of nodes: 1" backend=gce component=scaler dryRun=false
time="2022-07-10T19:49:18+03:00" level=debug msg="is working hours now: false - cron: * 5-17 * * 0-5" backend=gce component=scaler dryRun=false
time="2022-07-10T19:49:18+03:00" level=debug msg="at or under minimum nodes. will adjust to the minimum if needed" backend=gce component=scaler dryRun=false