cdwdirect/sos_flow

API addition: "stream handle" to support massive event traces.

Opened this issue · 0 comments

SOS "pub handle" is not optimized for capturing VERY LARGE amounts of streaming event trace data.

Proposed additional "stream handle" that is engineered to optimize for MANY small simple data chunks, rather than a few large, complex, self-describing events.

Assuming that events arise from fixed annotations in code that do not change filename, line number, type (start/stop) or semantics, those details can be captured once and stored in a lookup table.

descriptions

Trace events can then be stored as the event's id, a timestamp, a relation_id, a parent_relation_id (to capture traces of recursive functions), etc.

events

This will dramatically cut down on the metadata overhead for logging event traces using SOS.