jlewi/foyle

Make evaluation work with continuous logs analysis

Closed this issue · 1 comments

Per #84 and #139 we want to continuously process and ingest logs.
How do we update evaluation to work with this? There's a couple key questions

  1. Where is the Analyzer actually running?
  2. How does the evaluator know when the logs have been processed and can run steps that require them?

The Analyzer needs to access the pebble databases. The pebble databases can't be multi-process. So if evaluation is reusing logs/databases used by the active instance of Foyle then the evaluation binary can't run its own Analyzer.

I think it makese sense to give each experiment its own set of logs and DBs so we don't pollute the logs/DBs of the production instance.

The experiment process could then run its own Analyzer.