Reduce echo

[miguelwon]

Any advice on how to reduce echo?
I'm using the following pipeline applied to the webrtc_sendrecv example and I ear a very strong echo:

PIPELINE_DESC = '''
webrtcbin bundle-policy=max-bundle name=sendrecv stun_server=stun://stun.l.google.com:19302 turn_server=turn://user:root@18.185.76.156:3478
v4l2src ! video/x-raw,width=858,height=480,framerate=30/1 !
videoconvert ! queue max-size-buffers=1 ! x264enc bitrate=800 speed-preset=superfast tune=zerolatency key-int-max=15 !
video/x-h264,profile=constrained-baseline ! queue max-size-time=100000000 ! h264parse !
rtph264pay config-interval=-1 name=payloader !
application/x-rtp,media=video,encoding-name=H264,payload=96  ! sendrecv.
alsasrc device=hw:1,0 ! audioconvert ! audio/x-raw, format=S16LE, channels=2 ! audioresample ! queue ! opusenc ! rtpopuspay ! queue !
application/x-rtp, media=audio, channels=2, encoding-name=OPUS, payload=97 ! sendrecv.
'''

[thaytan]

You can use the webrtcdsp plugin to do echo cancellation - https://gstreamer.freedesktop.org/documentation/webrtcdsp/webrtcdsp.html

[miguelwon]

Thanks. Try it but didn't improve very much.

[thaytan]

Another thing you might want to do is reduce the default buffer-time of the alsasrc. The default is a conservative 200ms of ringbuffer, which is good for recording but not so much for real-time audio, nor for echo cancellation

[miguelwon]

Ok thanks. In fact, I notice that when using real-time audio I get higher latencies. Perhaps it is related with that. I'll try it and report. Thanks.

[thaytan]

You'll need to reduce the configured buffer size in both the audio source, and any sink that you add - and make sure that insert both halves of the echo cancellation plugin - on the capture side, and on the playback chain. We should add an example for that.

[miguelwon]

So, I'm still stuck on this. This is my setup: Raspi Pi 4 with Raspbian Buster and GStreamer 1.16 (from gst-build).

In the Raspi Pi 4 I have a speaker and mic and I'm testing the webrtc_sendrecv.py example with following pipeline (so sound only):

PIPELINE_DESC = '''
webrtcbin name=sendrecv stun-server=stun://stun.l.google.com:19302 bundle-policy=max-bundle
alsasrc device=hw:1,0  buffer-time=30000 ! audioconvert ! webrtcdsp ! webrtcechoprobe ! audioconvert ! queue ! opusenc ! rtpopuspay !
application/x-rtp,media=audio,encoding-name=OPUS,payload=96 ! sendrecv.
'''

Inspired by this discussion, I edited the decode bin section of webrtc_sendrecv.py to include webrtcechoprobe:

            q = Gst.ElementFactory.make('queue')
            conv1 = Gst.ElementFactory.make('audioconvert')
            webrtcechoproben = Gst.ElementFactory.make('webrtcechoprobe')
            conv2 = Gst.ElementFactory.make('audioconvert')
            resample = Gst.ElementFactory.make('audioresample')
            sink = Gst.ElementFactory.make('alsasink')
            self.pipe.add(q,conv1,webrtcechoproben,conv2, resample, sink)
            self.pipe.sync_children_states()
            pad.link(q.get_static_pad('sink'))
            q.link(conv1)
            conv1.link(webrtcechoproben)
            webrtcechoproben.link(conv2)
            conv2.link(resample) 

Even so the echo remains, meaning that when I send audio from browser to Raspi speaker, the Raspi mic sends back the initial audio (to the browser). I think that it could be related to the fact that webrtcdsp does not connect to webrtcechoprobe of the decodebin section, but with the webrtcechoprobe that follows it at PIPELINE_DESC. But if I remove webrtcechoprobe from PIPELINE_DESC, the script hangs right at the start of execution and no connection is established. Additionally, I haved tested the equivalent full pipeline in the Raspi Pi, which works quite ok with echo being cancelled:

gst-launch-1.0 alsasrc device=hw:1,0 buffer-time=100000 ! audioconvert ! webrtcdsp ! audioconvert ! queue ! opusenc ! rtpopuspay ! decodebin ! audioconvert ! webrtcechoprobe ! audioconvert ! audioresample ! alsasink

[thaytan]

You should not have 2 webrtcechoprobe elements. The idea is that you have a webrtcechoprobe before your audio playback sink, which takes note of what you're about to play (the audio you're receiving from the far end). The webrtcdsp element then processes your microphone audio to remove any echo from hearing your own playback.

If, on the other hand, you're getting your own audio echoed back by the remote peer there's not much you can do - the remote side needs to do some echo cancellation as well.

[ystreet]

These examples have moved to https://gitlab.freedesktop.org/gstreamer/gst-examples/-/tree/master/webrtc and this repository will not be updated further.