browsermt/bergamot-translator

Cache improvements

jerinphilip opened this issue · 0 comments

We have some feedback about cache usage from translateLocally (XapaJIaMnu/translateLocally#75).

@jerinphilip expects to take on jerinphilip/tagtransfer#3 at some point to see how useful the cache in a simulated link-surfing browsing workflow, with a bit of back and forth if there is time eventually.

The following was discussed at python merge and will need addressing at some point.

  1. Some workloads don't want cache on for the entire duration of work being provided. This can be managed through a Ptr, or more suitably after C++17 - std::optional.
  2. Per request/response cache configuration is possible and can be added. However I think it will be just enough to configure at start for whole of request/response.
  3. Certain configuration items are not ideal to be exposed outside, but rather be configured here as a function of other config items (eg: cache-mutex-buckets).
  4. Defaults driven by the downstream uses that make most sense, for python at least.

There some repetition eg: ternary operator used similar to

TranslationCache *cache = config_.cacheEnabled ? &cache_ : nullptr;
in several places can be changed into an inline function.