shotover/shotover-proxy

Coalesce transform - implement true flush_when_millis_since_last_flush

Opened this issue · 0 comments

rukai commented

Previously shotover's design did not allow for a true implementation of flush_when_millis_since_last_flush.
There was no way for the transform to trigger a flush all on its own, it had to wait for a new request to come in before the flush could be triggered.

This results in easily triggered poor behavior, for example:

  • flush_when_millis_since_last_flush set to 1000 (1 second)
  • client sends 1 request
  • 1 hour passes and no more requests are sent
  • flush never occurs and request is just stuck in shotover.

However shotover now supports this kind of functionality via force_run_chain.
So we should reimplement the flush_when_millis_since_last_flush logic of Coalesce to setup a tokio task in the background that will trigger a flush after the time has passed.

This should be done by:

  • setting up the task in CoalesceConfig::get_builder
  • the task runs a tokio::sleep(Duration::from_millis(flush_when_millis_since_last_flush)).await in a loop.
    • whenever the sleep elapses: trigger force_run_chain and send a message to the transform instance to let it know it needs to flush.
  • every time a flush occurs send the task a message to tell it to restart its timer