datalust/seq-api

Count of events in the result set before returning a partial result

Closed this issue · 1 comments

Hello there.

In the documentation is says this:

Seq internally limits the resources a query is allowed to consume.
:
:
If the result set is expected to be small, ListAsync() will buffer results and return a complete list:

What is small?

What is the limit to the number of events returned before you get a partial result?

Thanks.

Thanks for getting in touch. That comment is referring to how much memory will be used client-side to buffer the whole result set before the list is returned.

The ListAsync() method will internally watch for partial result sets and keep calling the Seq API until the desired number of events is reached; during this process, it'll add them internally to a list, which will grow until the whole requested result set has been retrieved.

"Too big" then really depends on how big the client machine is - if you're expecting more than a few hundred results, using the pattern from the README is probably worthwhile.

Seq used to use various metrics to decide when to return a partial result set (total scanned events, events-since-last hit), but since 2020 it internally uses timings for these, so although the parameters to the API can influence the duration of the timeout, there isn't any reliable way on the client-side to specify precisely when a partial result set should be returned.

Hope this helps!
Nick