python-streamz/streamz

Time based lookback window?

anovv opened this issue · 1 comments

anovv commented

Is there such a stream? I want to hold all of the events appeared within a giving time range starting from now (and do some aggregation downstream), e.g. all of the events for the last 15 mins. I found timed_window and sliding_window, but they have different purposes from what I see.

I believe timed_window is exactly what you want. Every specified time period, it emits all the events that have been seen in that period as a tuple, and then resets its internal buffer.

In [43]: s = streamz.Stream.from_periodic(lambda: 1, 1)

In [44]: s2 = s.timed_window(5)

In [45]: s2.sink(print)
In [46]: s.start()

[1, 1, 1, 1, 1]
[1, 1, 1, 1, 1]