Capture Latency b/w KVS and Consumer

[chrisfabri]

I would like to capture the latency a fragment experiences as it is sent from KVS, over the wire, and then parsed in the consumer.

I see there is millisBehindNow - but this represents the difference b/w the producer.putMedia event timestamp and the consumer.getMedia event timestamp, where each time stamp is taken on the KVS server at the time of the event.

once a record is received and parsed by the consumer, if a timetamp were taken at the correct point in time, then the difference b/w the server side timestamp and the timestamp taken in the consumer would represent this 'send and parse' latency.

With this new latency, the overall latency would then be millisBehindNow + kvsToConsumerLatency.

Looking at the FragmentMetadataVisitor.java class and specifically the stateMachineVisitor...I see that the millisBehindNow is being snapped with every endMasterElement encountered. But looking at the records themselves, the server time stamp is supplied at the beginning of each segment only.

I've built a visitor that follows the same state machine logic found in FragmentMetadataVisitor.StateMachineVisitor.
It snaps a timetimestamp when the endmaster record determines its reached the end of the segment.

Because of the way the code is structured and how communication b/w threads is setup currently, the time captured with this metric will reflect

1. send time from KVS to consumer for the fragment
2. record parsing time for the fragment
3. time spent sending data frame bytes to consuming thread

Looking at the MVK records for a single fragment, I see the following elements in the following order:

<<<<<---- ServerSide TS is taken

EBML
Segment

• Info
• Tracks
• Tags. (these contain FragNum/ServerTS/ProducerTS).
• Cluster
  -- SimpleBlock (50+ data frames)
• Tags (VoiceConnectorID)
• Tags.(TransactionId)
• Tags (CallID)
• Tags.(MillisByindNow/Token)

<<<<<---- Consumer side Timestamp2 is taken, after fragment is fully processed in KVS processing thread

The metric is then calculated as

• kvsToConsumerLatency = Timestamp2 – ServerSideTS

I’ve set up a visitor to calculate this value.