Is there any way to create insertable streams?
petergerten opened this issue · 3 comments
As both chrome and firefox now support the new insertable stream API, I was wondering if there is any way to process insertable streams in gstreamer. (https://www.w3.org/2019/09/18-mediaprocessing-harald-insertable-media-processing.pdf)
Yes - you can pretty much do any processing you like on a buffer between the encoder and payloader by writing a custom element, or using buffer probes as long as the payloader can still make some sense of the packets - and vice versa when receiving.
@thaytan that's great to hear.
I have a frame and frame-synchronised meta data in an OpenCV python environment.
Currently I send the frame to 'appsrc' which then passes it to x264enc and on to the payloader.
Any hints on how I would inject the metadata into the pipeline? As it would be needed after x264enc.
There's no trivial way, but the "normal" method of carrying such data in H.264/H.265 is by custom SEI packets. In the past, I've done this by attaching a custom Meta to the buffers, and using a custom element after the encoder to inject application specific SEI into the H.264 stream - and a corresponding extractor on the receiver side