Audio ideas?
Closed this issue · 39 comments
Anyone know ffmpeg we’ll enough to bring audio to life on the plugin?
try this:
add the audio source to videoConfig[source] after the video source
change the audio mapping
ffmpegCommand+= ' -map 0:1'
to
ffmpegCommand+= ' -map 1:1'
yes exactly, add the audio url after the video url with -i ... (always beware of not putting double spaces into the string)
can you attach ffmpeg debug output?
mmm... the debug output from ffmpeg is not there, only the launch command...
I think you have to first fix this bug
change
ffmpeg.stderr.on('data', function(data) {
into
ffmpeg.stderr.on('data', (data) => {
because "this" inside "function" is not bound correctly and this.debug is undefined
Good, now ffmpeg should be your friend!
Has something been cut in your output or simply ffmpeg hangs?
I see the first input listed but not the second...
I suggest you to fine tune the ffmpeg command-line separately, you just need to get the two inputs correctly recognized so you have to test and adjust only the first part of the command
ffmpeg -re -f mjpeg -i http://xxx:xxx@192.168.0.69/bha-api/video.cgi -i http://xxx:xxx@192.168.0.69/bha-api/audio-receive.cgi
First thing that comes to mind is to add format specification to the audio url, i.e. use -f option before -i
Sorry but I currently don't have a doorbird at hand to try myself.
ok now you have to tweak ffmpeg options...
after -f you have to specify the audio input format (like -f mjpeg for video) do you know which format doorbird uses?
According to https://www.doorbird.com/downloads/api_lan.pdf it is G.711 μ-law.
it could be G.711, their http api is similar to my DLINK webcam
try "-f mulaw"
btw it seems that doorbird support also the usual "rtsp" video/audio interface, why don't you use that?
rtsp://<device-ip>:<device-rtsp-port>/mpeg/media.amp
yes! just
ffmpeg -re -i rtsp://xxx:xxx@192.168.0.69/mpeg/media.amp
take a look at ffmpeg output, you should see the two streams 0:0 video and 0:1 audio
Does the method above work now?
I’m not able to try myself at the moment but I have a DoorBird also.
you have to revert this mod also
ffmpegCommand+= ' -map 1:1'
back to
ffmpegCommand+= ' -map 0:1'
but I still can't see input stream 0:1 in your log
missing -i after "-f mulaw"
I suggest to study ffmpeg a little... otherwise it wil be very difficult to tune the command, not every camera works out of the box!
have you tried rtsp again with "map" mod revert?
as you can see from the log there's an error receiving the video stream so ffmpeg exits without further processing
[udp @ 0000017fc9e5c600] 'circular_buffer_size' option was set but it is not supported on this build (pthread support is required)
it seems that your ffmpeg is not correctly built or otherwise there's a bug into ffmpeg itself (not so uncommon)
your version looks very strange to me
ffmpeg version N-91330-ga990184007
mine says
ffmpeg version 4.0.2
Good news is now I can see your separate audio stream, I'm sorry but you have to change the audio mapping
ffmpegCommand+= ' -map 1:1'
to
ffmpegCommand+= ' -map 1:0'
because it has to match exactly the stream number you see in the log
Stream #1:0: Audio: pcm_mulaw, 44100 Hz, mono, s16, 352 kb/s
libfdk_aac is missing too!
without libfdk_aac audio doesn't work at all..
Finally, recompiled ffmpeg with the https://github.com/jb-alvarado/media-autobuild_suite project. Did a full compile with everything (just because..)
Audio is now streaming in with the video, no more errors in ffmpeg (very cool, although seems to be white noise rather than the actual audio), but it makes a nice burst of audio then goes very quiet, not sure is something is cancelling audio or not. Stream kills itself after at 40 seconds (seen that before?). I've noticed ffmpeg continues to take a stream of audio even when the video stream has been stopped.
@llemtt I need to set audio to 8000hz, tried a few overrides but its stuck on 16Khz, were should this be changed?
Sample output:
[2018-8-27 15:53:34] [DoorBird] Start streaming video from DoorBird with 1280x720@299kBit
ffmpeg -re -f mjpeg -i http://xx:xxx@192.168.0.69/bha-api/video.cgi -f mulaw -i http://xxx:xxx@192.168.0.69/bha-api/audio-receive.cgi -map 0:0 -vcodec libx264 -pix_fmt yuv420p -r 10 -f rawvideo -tune zerolatency -vf scale=1280:720 -b:v 299k -bufsize 299k -maxrate 299k -payload_type 99 -ssrc 12243459 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params FUQTH2FdRBSzTNx5cZLCx2DuavqG1ewkfJ2gO46S srtp://192.168.0.34:53572?rtcpport=53572&localrtcpport=53572&pkt_size=1316 -map 1:0 -acodec libfdk_aac -profile:a aac_eld -flags +global_header -f null -ar 16k -b:a 24k -bufsize 24k -ac 1 -payload_type 110 -ssrc 14894808 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params cJQ02RHWdwbS6AgUaFnfGuUxSMFUCQlq3SSJibk9 srtp://192.168.0.34:60178?rtcpport=60178&localrtcpport=60178&pkt_size=1316
ffmpeg version N-91679-g4d87cd2882 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 8.2.0 (Rev1, Built by MSYS2 project)
configuration: --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --disable-debug --enable-ffnvcodec --enable-nvdec --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwavpack --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libaom --enable-version3 --enable-chromaprint --enable-decklink --enable-frei0r --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libfdk-aac --enable-libflite --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libkvazaar --enable-libmodplug --enable-libopenh264 --enable-libopenmpt --enable-librtmp --enable-librubberband --enable-libssh --enable-libtesseract --enable-libxavs --enable-libzmq --enable-libzvbi --enable-opengl --enable-libvmaf --enable-libcodec2 --enable-libsrt --enable-ladspa --enable-openssl --extra-cflags=-fopenmp --extra-libs=-lgomp --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DLIBSSH_STATIC --extra-ldflags='-Wl,--allow-multiple-definition' --extra-cflags=-DCACA_STATIC --extra-cflags=-DMODPLUG_STATIC --extra-cflags=-DCHROMAPRINT_NODLL --extra-libs=-lstdc++ --extra-cflags=-DZMQ_STATIC --extra-libs=-lpsapi --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads --extra-cflags=-DKVZ_STATIC_LIB --enable-nonfree
libavutil 56. 19.100 / 56. 19.100
libavcodec 58. 25.100 / 58. 25.100
libavformat 58. 17.103 / 58. 17.103
libavdevice 58. 4.101 / 58. 4.101
libavfilter 7. 26.100 / 7. 26.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100
Input #0, mjpeg, from 'http://xxx:xxx@192.168.0.69/bha-api/video.cgi':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 640x480 [SAR 96:96 DAR 4:3], 25 tbr, 1200k tbn, 25 tbc
[mulaw @ 0000028e16012d40] Estimating duration from bitrate, this may be inaccurate
Guessed Channel Layout for Input Stream #1.0 : mono
Input #1, mulaw, from 'http://xxx:xxx@192.168.0.69/bha-api/audio-receive.cgi':
Duration: 00:03:46.76, bitrate: 352 kb/s
Stream #1:0: Audio: pcm_mulaw, 44100 Hz, mono, s16, 352 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Stream #1:0 -> #1:0 (pcm_mulaw (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
[swscaler @ 0000028e163ab000] deprecated pixel format used, make sure you did set range correctly
[libx264 @ 0000028e162b0d00] using SAR=3/4
[libx264 @ 0000028e162b0d00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0000028e162b0d00] profile High, level 3.1, 4:2:0, 8-bit
Output #0, rtp, to 'srtp://192.168.0.34:53572?rtcpport=53572&localrtcpport=53572&pkt_size=1316':
Metadata:
encoder : Lavf58.17.103
Stream #0:0: Video: h264 (libx264), yuv420p, 1280x720 [SAR 3:4 DAR 4:3], q=-1--1, 299 kb/s, 10 fps, 90k tbn, 10 tbc
Metadata:
encoder : Lavc58.25.100 libx264
Side data:
cpb: bitrate max/min/avg: 299000/0/299000 buffer size: 299000 vbv_delay: -1
[mulaw @ 0000028e16012d40] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
Output #1, rtp, to 'srtp://192.168.0.34:60178?rtcpport=60178&localrtcpport=60178&pkt_size=1316':
Metadata:
encoder : Lavf58.17.103
Stream #1:0: Audio: aac (libfdk_aac) (ELD), 16000 Hz, mono, s16, 24 kb/s
Metadata:
encoder : Lavc58.25.100 libfdk_aac
frame= 4 fps=0.0 q=39.0 size= 16kB time=00:00:01.19 bitrate= 109.7kbits/s dup=0 drop=1 speed=2.33x
frame= 4 fps=3.9 q=39.0 size= 16kB time=00:00:01.29 bitrate= 101.6kbits/s dup=0 drop=1 speed=1.28x
frame= 4 fps=2.6 q=39.0 size= 16kB time=00:00:01.35 bitrate= 96.8kbits/s dup=0 drop=1 speed=0.888x
frame= 4 fps=2.0 q=39.0 size= 16kB time=00:00:01.45 bitrate= 90.4kbits/s dup=0 drop=1 speed=0.711x
frame= 4 fps=1.6 q=39.0 size= 16kB time=00:00:01.55 bitrate= 84.8kbits/s dup=0 drop=1 speed=0.609x
frame= 4 fps=1.3 q=39.0 size= 16kB time=00:00:01.61 bitrate= 81.4kbits/s dup=0 drop=1 speed=0.528x
frame= 4 fps=1.1 q=39.0 size= 16kB time=00:00:01.71 bitrate= 76.9kbits/s dup=0 drop=1 speed=0.48x
frame= 4 fps=1.0 q=39.0 size= 16kB time=00:00:01.80 bitrate= 72.8kbits/s dup=0 drop=1 speed=0.443x
Sorted the -ar option but this will cause the stream to crash, assume bitrate has to change down?
@llemtt I need to set audio to 8000hz, tried a few overrides but its stuck on 16Khz, were should this be changed?
Homekit supported audio sample rates are 16kHz and 24kHz only, why do you want to change to 8000Hz?
Did anyone ever get this working?
After speaking to Doorbird about Audio, the current API is not for general use, seems more targeted to other systems e.g. Loxone. Unless someone can see differently
@llemtt we have audio receive working now on the plugin https://github.com/brownad/homebridge-doorbird
what is the simplest way to support 2way with the microphone?
The API docs are revealing more nowadays
https://www.doorbird.com/downloads/api_lan.pdf?rev=0.27