aws/amazon-kinesis-video-streams-parser-library

Getting data when the kvs stream ends is time consuming

Closed this issue · 5 comments

kpatil001 commented

Hi Team,
I am working on use case, where we are performing transcription on audio call which is streamed from a telephony system to KVS, we are using amazon-chime-voiceconnector-transcription to perform this, which is internally using kinesis parser library. I am observing that when a call ends i.e, when KVS stream is about to end, fetching data from KVS is taking 3000+ milliseconds of time. getByteBufferFromStream() is used to get data from kvs, this function is called here.

In following logs I am logging last two call to request function.

Three numbers logged beside getByteBufferFromStream() function call are :

  • 1st number is the time taken by getByteBufferFromStream() function
  • 2nd number is the value of demand
  • 3rd number is the length of ByteBuffer returned by getByteBufferFromStream()
2021-05-07 12:06:56.460 INFO  KVSByteToAudioEventSubscription:62 - Call to request function :: demand : 12 
2021-05-07 12:06:56.461 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   2 16 1280
2021-05-07 12:06:56.624 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   162 15 1280
2021-05-07 12:06:56.627 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   1 14 1280
2021-05-07 12:06:56.805 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   177 13 1280
2021-05-07 12:06:56.819 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   10 12 1280
2021-05-07 12:06:56.821 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   1 11 1280
2021-05-07 12:06:56.988 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   166 10 1280
2021-05-07 12:06:56.990 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   1 9 1280
2021-05-07 12:06:57.169 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   177 8 1280
2021-05-07 12:06:57.171 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   1 7 1280
2021-05-07 12:06:57.173 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   1 6 1280
2021-05-07 12:06:57.432 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   258 5 1280
2021-05-07 12:06:57.434 INFO  KVSByteToAudioEventSubscription:62 - Call to request function :: demand : 12 
2021-05-07 12:07:00.898 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   3465 16 960
2021-05-07 12:07:00.900 INFO  KVSByteToAudioEventSubscription:73 - Time n size :: getByteBufferFromStream   0 15 0

As you can see that last 2nd call to getByteBufferFromStream is taking 3465 milliseconds and returns butebuffer of 960 length, sometimes length of byteBuffer is 0. Could you please provide some pointers on the cause of this latency and if there is a way to minimize or avoid this latency.

shearn89 commented

I think I'm bumping into this - during testing, if the stream has ended when I invoke my lambda, it takes just over 3 seconds, where normally it takes a couple of hundred ms warm. I suspect there's a 3 second timeout on the call, and my lambda then says 'got no data, returning'.

stefankiesz commented

Hello, apologies for the late response, it seems that this is related to the 3 second timeout of GetMedia. It takes 3 seconds of no data before it returns. Is this still an issue we can assist you with?

shearn89 commented

Uh, I ran into this nearly 2 years ago. In the end I rewrote my entire function into Go and used a go MKV parser to read from the Kinesis stream. So no, I no longer need assistance.

For future googlers at least, is this 3 second timeout documented somewhere, or configurable?

stefankiesz commented

The 3 second timeout is not configurable, and we don't currently have this documented anywhere. We are considering adding these details to our GetMedia documentation, and I will link it here once that becomes available. Thank you for bringing this to our attention.