metarhia/metacom

Make stream size optional

Opened this issue · 1 comments

Is your feature request related to a problem? Please describe.

Currently we are able to stream files, so there is a size parameter provided. Using stream size Metacom can track incoming bytes and automatically end the stream, when it's done. To cover some cases, when the stream has to be infinite (e.g. data traffic) we need to allow users create streams without required size attached and make them able to end streams by themselves.

Describe the solution you'd like

Make stream size parameter as optional.

Describe alternatives you've considered

We can collect some data in memory and stream it as a binary array with known size.

Additional context

There is a caveat when a writable stream is stopped by nodejs readable calling end() method: stream end instruction on client is received before some chunks do, what makes a situation of a race condition over websocket resulting in small data loss. This problem must be investigated to make optional stream size feature safe.
As a testable workaround we can set timeout in 100-500ms in writable end method.

Problem investigated.
Writable end() works correct.
When we use transport.send with buffer and transport.send with parsed object, they are two different transport.send that work in parallel