apache/accumulo

Remove Cache's from CompactionJobGenerator

Closed this issue · 5 comments

TabletGroupWatcher.manageTablets creates a new CompactionJobGenerator each time the method is called (each iteration of processing the metadata table tablets for user tables, for example). This is likely done so that any changes in CompactionService configuration can be picked up on the next iteration, rather than having to respond to configuration changes. Evaluate whether the Caffeine Cache's in the CompactionJobGenerator are still needed.

It looks like a CompactionJobGenerator is also created in the TabletManagementIterator, so we end up creating Caffeine Cache's in the Manager and TabletServers that we likely don't need.

TabletGroupWatcher.manageTablets creates a new CompactionJobGenerator each time the method is called (each iteration of processing the metadata table tablets for user tables, for example)

Looking at the caches, I think they are still useful in this situation. Lots of tablets pass through the CompactionJobGenerator after its created and the caches help avoid redoing work for each tablet. I am not sure what the implication of constantly recreating the caches are for the metrics code though.

From the description it does not seem that cache metrics for this function would provide much information to the end users - would they even have way to adjust it? Possibly higher level, tracking jobs would provide better insight?

From the description it does not seem that cache metrics for this function would provide much information to the end users - would they even have way to adjust it? Possibly higher level, tracking jobs would provide better insight?

It looks like metrics is already disabled for these Caches.

Looking at the caches, I think they are still useful in this situation.

Closing this based on this comment.