deitch/aws-asg-roller

asg-roller doesn't honor ROLLER_KUBERNETES setting when handling non-kubernetes nodes

chroche opened this issue · 4 comments

chroche commented

Hi,

I'm trying to use aws-awg-roller to update a number of non-EKS auto-scaling groups. The aws-awg-roller itself is running on EKS using latest DockerHub image 6d574cd7d46b726b7a60edae2914ee577d4c947b. I set the following environment in the Pod definition:

        - name: ROLLER_KUBERNETES
          value: "false"

However I'm getting the following error messages in the logs, indicating that aws-asg-roller is still trying to access the k8s API to get info about the EC2 instances it manages:

2020/12/12 15:13:46 [parser-new-nonprod-2] ok
2020/12/12 15:13:46 [parser-new-nonprod-2] Unable to update node annotations: Unexpected error getting kubernetes node : nodes "ip-10-169-188-248.ec2.internal" not found

2020/12/12 15:18:04 Unable to set disabled scale down annotations: Unexpected error getting kubernetes node : nodes "ip-10-169-184-139.ec2.internal" not found

2020/12/12 15:18:04 [logger-new-nonprod-2] error calculating adjustment - skipping: unexpected error readiness handler terminating node ip-10-169-190-85.ec2.internal: Unexpected error getting kubernetes node : nodes "ip-10-169-190-85.ec2.internal" not found

Regards,
Chris

deitch commented

Well, if nothing else, you found a bug in the logging. But the core issue is the bigger one.

deitch commented

It looks like it is checking it much later than it should. Interesting catch. Hang tight.

deitch commented

Ah, no, not quite. It actually need to check if it is in Kubernetes first, since the default is different when running inside Kubernetes (true) vs not (false). The problem is that it is not respecting the false correctly.

deitch commented

And that isn't it either, but I have it now.