improve the behavior of needs_merge

Question

improve the behavior of needs_merge

Closed this issue 10 years ago · 3 comments

A few things here:

We shouldn't even call needs merge outside of merge windows, as it's pointless, repeated work and blocks the vnode, raising tail latencies.
We should add some randomness to the scheduling of needs merge to make lockstep less likely.
Reconsider the 3 minute needs_merge interval. We should at least make it tunable.

Answer 1 · 2013-11-07T20:02:32.000Z

From #42, we should also make sure that it's possible to stop the merge worker from doing anything before kv starts up, if possible in a generic way.

Answer 2 · 2014-05-12T17:09:11.000Z

Going to go ahead and close this, since I think that @engelsanchez's latest changes cover this one. @engelsanchez please reopen and move to 2.1 if there's still work to be done.

Answer 3 · 2014-05-12T17:22:52.000Z

I had left it open because my change doesn't address not calling needs merge outside of the merge window. But given that it sounds like a marginal optimization anyway, I'm fine with leaving this closed and re-open in the future if latency tuning says it should be changed.