aws/aws-node-termination-handler

Fail the app on missing queue

LiorRabin opened this issue · 3 comments

LiorRabin commented

Describe the feature
Fail the app on missing queue if queue mode is enabled.

Is the feature request related to a problem?
We're using NTH with enableSqsTerminationDraining set to true
If we don't specify the queueURL correctly (or there is no queue in AWS) the application logs a warning but keeps being healthy.
Why not fail the app in this case?

github-actions commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want this issue to never become stale, please ask a maintainer to apply the "stalebot-ignore" label.

cjerad commented

Hi @LiorRabin

I deployed NTH with a bogus SQS queue URL and the logs indicated that NTH did shutdown

2024/01/29 19:28:42 INF Started monitoring for events event_type=SQS_MONITOR
2024/01/29 19:28:44 WRN There was a problem monitoring for events error="AWS.SimpleQueueService.NonExistentQueue: The specified queue does not exist." event_type=SQS_MONITOR
2024/01/29 19:28:46 WRN There was a problem monitoring for events error="AWS.SimpleQueueService.NonExistentQueue: The specified queue does not exist." event_type=SQS_MONITOR
2024/01/29 19:28:48 WRN There was a problem monitoring for events error="AWS.SimpleQueueService.NonExistentQueue: The specified queue does not exist." event_type=SQS_MONITOR
2024/01/29 19:28:50 WRN There was a problem monitoring for events error="AWS.SimpleQueueService.NonExistentQueue: The specified queue does not exist." event_type=SQS_MONITOR
2024/01/29 19:28:50 WRN Stopping NTH - Duplicate Error Threshold hit.
panic: AWS.SimpleQueueService.NonExistentQueue: The specified queue does not exist.

I think the Pod's restartPolicy is causing the NTH container to be restarted.

LikithaVemulapalli commented

We were not able to replicate the issue and NTH did shutdown from the logs, closing this issue. Thanks!