Feature: Flush destination every Nth record or every X seconds

Question

Feature: Flush destination every Nth record or every X seconds

Closed this issue 2 years ago · 3 comments

Feature description

Right now Flush is used only when a destination stops. If the destination is writing records asynchronously (i.e. destination implements WriteAsync) the SDK should call Flush repeatedly every Nth record or every X seconds (exact numbers to be determined when implementing behavior) so the developer does not have to implement this behavior themselves.

Answer 1 · 2022-06-03T15:21:21.000Z

Not sure if this is worth addressing if we are going to change the Destination interface (see ConduitIO/conduit#434).

Answer 2 · 2022-06-06T18:14:40.000Z

Not sure if this is worth addressing if we are going to change the Destination interface (see ConduitIO/conduit#434).

Agreed. 👍

Answer 3 · 2022-06-27T17:15:20.000Z

Closing in favor of ConduitIO/conduit#434