ExpediaGroup/adaptive-alerting

Avoid throwing exception when trying to create a metric and it already exists

Closed this issue · 2 comments

In MetricEventHandler, if a metric already exists in the database, the handler throws an exception to avoid trying to reinsert it. At scale, this will happen continuously. We have a handler wrapping that to avoid filling the log with exceptions.

We would like to avoid throwing exceptions in what is definitely not an exceptional situation. The main reason is that all metrics enter through the front door, and if there are easy ways to avoid throwing exceptions continuously then it seems like we should adopt them. https://stackoverflow.com/questions/567579/how-expensive-are-exceptions

(I understand the whole concept of premature optimization, but doing the analysis here would take more effort than simply replacing the exception-based control flow with a non-exception-based control flow.)

Suggestion would be to provide an overriding implementation of the JPA MetricRepository.create() method that performs an existence check and then inserts only if the metric doesn't exist.

Alternatively we could do implement this functionality in the DB itself (create-if-absent sort of thing), but this is less preferable since it would tie us to a particular DBMS.

Aside: It was Knuth who famously wrote, "premature optimization is the root of all evil." But the context where he said that, he was talking about how there's 97% of the program that will never need optimization, and 3% that will need it. He was saying that it's wasteful to guess at what's in the 3% while you're still coding. In this case, doing an expensive and unnecessary operation in normal use seems like an obvious part of the 3%.

Closing, as we are removing metrics from the database.