HubSpot/Singularity

Missing documentation for reworked cooldown logic

bydga opened this issue · 3 comments

bydga commented

Hi, we trying to upgrade our Singularity installation (from way too old 0.15.1) to 0.23.0 (we will proceed to 1.0 after that). However, after the upgrade, when I schedule a cron that always crashes to run every 5 mins, he enteres cooldown after the first 3 crashes (cooldownAfterFailures: 3), but then he exits and enteres the cooldown every few minutes (https://prnt.sc/p1b7w9). cooldownExpiresAfterMinutes is set to 300, so that shouldn't be the problem.

Might it be related to the recent cooldown reworks?
#1961
#1973

bydga commented

yes, its the fast vs slow cooldown configuration - which was implemented, however not reflected in the docs. and im not sure how the fast cooldown should work with the scheduled requests (crons)

Updated docs in #2015 , sounds like the defaults may be a bit too strict for this case, and increasing fastCooldownExpiresMinutesWithoutFailure/slowCooldownExpiresMinutesWithoutFailure will return you to normal behavior

bydga commented

Yep, u r right, setting those config vals fixed my problem. 👍