example does not work with raspbian stretch lite image
alexheinz opened this issue · 14 comments
Hello,
I have tried to run a rpicamsrc example gstreamer pipeline taken from https://github.com/thaytan/gst-rpicamsrc/wiki with the latest raspbian stretch lite (no X) (2018-04-18-raspbian-stretch-lite.img) and it does not work.
I have installed raspbian, assigned 256MB of memory to the GPU and made the following changes
sudo aptitude update && sudo aptitude upgrade
sudo apt-get install gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-plugins-base gstreamer1.0-omx-rpi gstreamer1.0-omx-rpi-config autoconf automake libtool pkg-config libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libraspberrypi-dev git gstreamer1.0-tools mc python-gst-1.0 gstreamer1.0-alsa
git clone https://github.com/thaytan/gst-rpicamsrc.git
cd gst-rpicamsrc
./autogen.sh --prefix=/usr --libdir=/usr/lib/arm-linux-gnueabihf/
make
sudo make install
my /boot/config.txt:
pi@raspberrypi:~ $ cat /boot/config.txt | grep -v "#" | sort -u
disable_overscan=1
dtparam=audio=on
gpu_mem=256
start_x=1
The gstreamer pipelines are run via SSH and were copied from the rpicamsrc documentation
player (which is started before the sender):
gst-launch-1.0 udpsrc port=5200 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! omxh264dec ! autovideosink
sender:
gst-launch-1.0 rpicamsrc preview=0 bitrate=1500000 ! 'video/x-h264, width=1280, height=720, framerate=30/1,profile=high' ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=5200
The problem is that I do not see a picture on my screen with those pipelines.
this is the output of the player pipeline:
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0: Could not initialise Xv output
Additional debug info:
xvimagesink.c(1760): gst_xv_image_sink_open (): /GstXvImageSink:autovideosink0-actual-sink-xvimage:
Could not open display (null)
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Assuming that the missing X influences the behavior, I have tried to change autovideosink to fbdevsink because the following pipeline works:
gst-launch-1.0 videotestsrc ! fbdevsink
However gst-launch-1.0 udpsrc port=5200 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! omxh264dec ! fbdevsink does not work either.
here is the corresponding output:
pi@raspberrypi:~ $ gst-launch-1.0 udpsrc port=5200 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! omxh264dec ! fbdevsink sync=falseSetting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0: Internal data stream error.
Additional debug info:
gstomxvideodec.c(1599): gst_omx_video_dec_loop (): /GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
stream stopped, reason not-negotiated
Execution ended after 0:00:05.051817633
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Where is my mistake or is the example in the documentation broken?
Thank you for your help!
fbdevsink requires RGB or RGBA format data, and the decoder is outputting I420. Insert a videoconvert element before fbdevsink to convert (performance will be terrible), or you can try glimagesink instead and hopefully do more on the GPU directly.
@thaytan
Thank you for the quick feedback.
It looks like glimagesink requires X.
pi@raspberrypi:~ $ gst-launch-1.0 udpsrc port=5200 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! omxh264dec ! glimagesink
Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
ERROR: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: Failed to connect to X display server
Additional debug info:
gstglimagesink.c(1009): _ensure_gl_setup (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink
Setting pipeline to NULL ...
Freeing pipeline ...
Is there another alternative that does not require X?
How does omxplayer manage to show the video without X?
You can build glimagesink to not require X, but raspbian might not do that
Thank you for the explanation.
Although I do realize that this not a problem of rpicamsrc, I would highly appreciate help with compiling glimagesink without an X dependency on raspbian stretch lite.
Do you happen to know where I can find instructions for doing so?
There's some slight old-but-should-still-work instructions here: https://fhackts.wordpress.com/2015/12/11/compiling-glimagesink-for-gles-without-xorg-on-raspberrypi/
Just make sure you try and compile the same source / version as you've got installed, or you'll have to rebuild the whole GStreamer stack. 'gst-inspect-1.0 glimagesink' will tell you which things to get like this:
Plugin Details:
Name opengl
Description OpenGL plugin
Filename /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstopengl.so
Version 1.10.4
License LGPL
Source module gst-plugins-bad
i.e., you want the gst-plugins-bad 1.10.4 tarball there.
the instructions seem to be out of date (or do not apply to raspbian) because they refer to a few files that do not exist in the raspbian stretch lite image:
- /opt/vc/lib/libEGL.so
- /opt/vc/lib/libGLESv2.so
I have tried this way (without success):
apt-get source gst-plugins-bad1.0
cd gst-plugins-bad1.0-1.10.4/
./configure CFLAGS="-I/opt/vc/include -I /opt/vc/include/interface/vcos/pthreads -I /opt/vc/include/interface/vmcs_host/linux/" LDFLAGS="-L/opt/vc/lib" --disable-gtk-doc --disable-opengl --enable-gles2 --enable-egl --disable-glx --disable-x11 --disable-wayland --enable-dispmanx --with-gles2-module-name=/opt/vc/lib/libbrcmGLESv2.so --with-egl-module-name=/opt/vc/lib/libbrcmEGL.so
make -j 4
mkdir -p ~/.local/share/gstreamer-1.0/plugins
cp ./ext/gl/.libs/libgstopengl.so ~/.local/share/gstreamer-1.0/plugins
I have tried to run this:
GST_GL_WINDOW=dispmanx GST_GL_API=gles2 GST_GL_PLATFORM=egl gst-launch-1.0 videotestsrc ! glimagesink
... and this is the error message that I get:
Setting pipeline to PAUSED ... libEGL warning: DRI3: xcb_connect failed libEGL warning: DRI2: xcb_connect failed libEGL warning: DRI2: xcb_connect failed ERROR: Pipeline doesn't want to pause. Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0"; ERROR: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: Failed to initialize egl: EGL_NOT_INITIALIZED Additional debug info: gstglimagesink.c(1009): _ensure_gl_setup (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink Setting pipeline to NULL ... Freeing pipeline ...
any idea what to change?
Thank you for your help!
@alexheinz I have exactly the same problem! Did you have any success meanwhile?
@moritzvieli
Unfortunately I still have the same problem. Please let me know if you find a way to make this work.
@alexheinz Seems like you could get rid of the X requirement using the script mentioned by Swap-File in th every end:
https://gist.github.com/sphaero/02717b0b35501ad94863
I had to change minor things (apt-get update, fixed typo for dependencies mentioned in his script):
https://gist.github.com/moritzvieli/417de950209a24a4f7a57ce1bb5bfeb7
I have been able compiling it from source with this script. However, it still does not work.:
0:00:00.596053551 843 0x75943b20 WARN glimagesink gstglimagesink.c:1012:_ensure_gl_setup:<sink> error: Failed to initialize egl: EGL_NOT_INITIALIZED
0:00:00.596796405 843 0x75943b20 WARN playbin gstplaybin2.c:4663:autoplug_select_cb:<playbin0> Could not activate sink glimagesink
If you have more success, please let me know. :)
The root cause is this:
libEGL warning: DRI3: xcb_connect failed
libEGL warning: DRI2: xcb_connect failed
libEGL warning: DRI2: xcb_connect failed
But I could not get rid of it although creating many symbolic links from everywhere to everywhere...
@alexheinz Got it working with the following script:
https://gist.github.com/moritzvieli/417de950209a24a4f7a57ce1bb5bfeb7
After this, you have to create symlinks so gstreamer uses the correct libraries:
sudo ln -fs /opt/vc/lib/libbrcmEGL.so /usr/lib/arm-linux-gnueabihf/libEGL.so.1
sudo ln -fs /opt/vc/lib/libbrcmGLESv2.so /usr/lib/arm-linux-gnueabihf/libGLESv2.so.2
I don't know, why the flags with-gles2-module-name and with-egl-module-name don't work in the install script... but after this fix, I'm able to play videos on Raspbian Stretch with gstreamer.
@moritzvieli:
thank you VERY much! You made my day!
Also works for me with the latest raspbian stretch lite (2018-06-27-raspbian-stretch-lite.img)
The latency is better than expected with my setup:
picamera pointed at TV - first pi2 - ethernet - second pi2 - hdmi input 1 of TV
picamera pointed at TV - first pi2 - hdmi input 2 of TV
resolution: 1280x720
frame rate: 30 Hz
bitrate: 2Mbit/s
I have used the frame counter of rpicamsrc and took pictures of my TV screen:
encoding + hdmi to tv delay (measured by looking at the preview picture) is 3 frames, so ~100ms
end to end delay is 6 frames, so ~200ms
source:
gst-launch-1.0 -v rpicamsrc annotation-text-size=410 annotation-mode=0x00000200 rotation=180 preview=1 bitrate=2000000 ! 'video/x-h264, width=1280, height=720, framerate=30/1,profile=high' ! h264parse ! rtph264pay config-interval=10 pt=96 ! udpsink host=192.168.160.113 port=5200
sink:
gst-launch-1.0 udpsrc port=5200 ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! h264parse ! omxh264dec ! glimagesink
I am closing the issue because it has been solved. Thank you @moritzvieli and @thaytan for your help!
I have another question regarding the possible performance but I will create a new issue for it
I keep getting the following error when launch the sink command:
Setting pipeline to PAUSED ...
0:00:00.264954070 1239 0x49a940 WARN glimagesink gstglimagesink.c:1012:_ensure_gl_setup:<sink> error: glGetString not defined or returned invalid value
ERROR: Pipeline doesn't want to pause.
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";
ERROR: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: glGetString not defined or returned invalid value
Additional debug info:
gstglimagesink.c(1012): _ensure_gl_setup (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink
Setting pipeline to NULL ...
Freeing pipeline ...
I guess this is related to libEGL.so.1 and libGLESv2.so.2 not being correct, so I played a little with linking those libraries. But to no avail.
Anyone ran into this issue or has a clue?
Edit
Originally is started from the 2018-11-13-raspbian-stretch-lite image. Got it working now using the 2018-06-27-raspbian-stretch-lite image.