Latency
crossan007 opened this issue · 6 comments
currently trying to use rpicamsync in a multi-camera set up, but I'm seeing additional latency when using rpicamsrc instead of raspivid.
Working Non-rpicamsrc Scenario:
For example, when using the following pipelines, the video from both sources is rendered at the same time:
Source 1:
raspivid -t 999999 -h 720 -w 1280 -fps 25 -b 2000000 -o - | gst-launch-1.0 -v fdsrc do-timestamp=true ! h264parse ! matroskamux ! tcpclientsink host=192.168.10.15 port=20001
Sink 1:
raspivid -t 999999 -h 720 -w 1280 -fps 25 -b 2000000 -o - | gst-launch-1.0 -v fdsrc do-timestamp=true ! h264parse ! matroskamux ! tcpclientsink host=192.168.10.15 port=20001
Source 2:
gst-launch-1.0 v4l2src ! jpegparse ! videorate ! image/jpeg,framerate=30/1 ! queue ! matroskamux ! queue ! tcpclientsink host=192.168.10.15 port=20002
Sink 2:
gst-launch-1.0 tcpserversrc host=0.0.0.0 port=20002 ! matroskademux ! decodebin ! videoconvert ! autovideosink sync=false
Note the synchronized flashing
Laggy rpicamsrc scenario:
When using rpicamsrc in gsrteamer, the raspicam arrives late:
Source1:
gst-launch-1.0 rpicamsrc name=videosrc ! h264parse ! video/x-h264,framerate=30/1 ! queue ! matroskamux ! queue ! tcpclientsink host=192.168.10.15 port=20001
Sink 1:
raspivid -t 999999 -h 720 -w 1280 -fps 25 -b 2000000 -o - | gst-launch-1.0 -v fdsrc do-timestamp=true ! h264parse ! matroskamux ! tcpclientsink host=192.168.10.15 port=20001
Source 2:
gst-launch-1.0 v4l2src ! jpegparse ! videorate ! image/jpeg,framerate=30/1 ! queue ! matroskamux ! queue ! tcpclientsink host=192.168.10.15 port=20002
Sink 2:
gst-launch-1.0 tcpserversrc host=0.0.0.0 port=20002 ! matroskademux ! decodebin ! videoconvert ! autovideosink sync=false
Note the ~5 second delay from when Sink2 displays the light pulse until Sink1 displays the pulse:
Additional Observation
Changing the laggy scenario's sensor-mode to 2 yeilded slightly better results, but there is still a slightly under 1 second delay:
gst-launch-1.0 v4l2src ! jpegparse ! videorate ! image/jpeg,framerate=30/1 ! queue ! matroskamux ! queue ! tcpclientsink host=192.168.10.15 port=20002
In VoctoMix
The above examples are using ad-hoc instances of gstreamer. When used with my script at https://github.com/crossan007/PiCamFleetMapper in conjunction with https://github.com/voc/voctomix providing a synchronized clock and end-to-end preserved timestamps courtesy of matroska container, the issue is a little worse, and frames drop:
The obvious difference is that you're asking raspivid for 25fps 1280x720 and 2Mbit/s, but with rpicamsrc you only restrict the framerate (to 30fps, so that's also different) - so it'll be streaming 1920x1080 at 17Mbit/s 30fps. To confirm a problem, you should try with the same settings:
gst-launch-1.0 rpicamsrc name=videosrc bitrate=2000000 ! h264parse ! video/x-h264,framerate=25/1,width=1280,height=720 ! queue ! matroskamux ! queue ! tcpclientsink host=192.168.10.15 port=20001
P.S. I think your 'sink 1' pipeline is wrong in each case.
Ah good catch of my copy/pasta error 👍
Test results as suggested:
Source 1 (RasPi zero)
gst-launch-1.0 rpicamsrc name=videosrc bitrate=2000000 ! h264parse ! video/x-h264,framerate=25/1,width=1280,height=720 ! queue ! matroskamux ! queue ! tcpclientsink host=192.168.10.15 port=20001
Sink (i7-4790k + GTX970 / 16GB RAM)
gst-launch-1.0 tcpserversrc host=0.0.0.0 port=20001 ! matroskademux ! decodebin ! videoconvert ! autovideosink sync=false
Source 2 (RasPi 3 / Logitec C922X USB/MJPEG)
gst-launch-1.0 v4l2src ! jpegparse ! videorate ! image/jpeg,framerate=30/1 ! queue ! matroskamux ! queue ! tcpclientsink host=192.168.10.15 port=20002
Sink (i7-4790k + GTX970 / 16GB RAM)
gst-launch-1.0 tcpserversrc host=0.0.0.0 port=20002 ! matroskademux ! decodebin ! videoconvert ! autovideosink sync=false
It seems like this may be the solution... I should have seen that!
Hello,
please, could anyone help me? I am trying to stream fullHD resolution from Raspicam V2 via this gstreamer pipeline:
rpicamsrc ! video/x-h264, width=1920,height=1080,framerate=25/1 ! h264parse ! rtph264pay config-interval=1 name=pay0 pt=96
But the image is always cropped, it is possible to achieve this via v4l2src, but the stream is incredibly slow anyway this approach is bad, professionals suggest rpicamsrc, but how to achieve 1920x1080, the Sony chip is definitely capable of fullHD and raspivid can achieve this to output on screen, but I need to transfer it over UDP.
Thank you
@ihora I've never had success with rtph264pay - hence I always use matroskamux
I don't know why I had the issue, but I suggest you try matroska if possible
I guess rtph264pay is used for streaming over the network to encapsulate stream to packets, matroskamux is used when saving video to file.
According to documentation of Sony chip, the 1920x1080 video has a reduced Field Of View therefore I cannot expect the same field of view contrary to 1280x720.
My question is why raspivid output and rpicamsrc output at default that means 1920x1080 are not exactly the same as rpicamsrc is a wrapper for raspivid. Raspivid is pretty fast and has big field of view when requesting fullHD, maybe I am wrong and both are the same.