Question: delay in live camera streaming?
rogermeijer opened this issue · 13 comments
Dear,
I was wondering how the plugin is fetching the stream. Comparing the Nest App vs. Nest Web, I see a major difference in delay of the video (~1 sec vs. 5 sec). Which stream is the plugin using, because in HomeKit my stream also shows a delay of ~5 seconds. Is there a way to optimize this?
I am using a Raspberry Pi (3), and have ffmpeg installer ("ffmpegCodec": "h264_omx")
Thanks a lot!
I reverse-engineered the Nest Website to get the plugin operational. The delay you see is due to your Raspberry Pi having to encode the web video into an iOS-compatible stream. The faster your device, the faster the encode. The only thing you could do currently is run Homebridge on a more powerful machine, but I'm looking into removing the need to encode. The Nest app uses the same endpoint as the website but somehow asks the server for an iOS-compatible stream.
After doing more research into this, it seems that the Nest app and the Nest Website actually decode the stream on the device itself (e.g. iPhone or Desktop). Unfortunately, I cannot tell the Home app how to correctly decode the stream, so the stream must be sent already in the correct format from the device running Homebridge. This means the stream must be encoded before being sent. For now, I'm going to close this issue since nothing can be done, but I'll link it to the README because it's a good question.
Thank you for the prompt and clear reply. I understand the issue, indeed something you can't solve on application level. I will look at the performance of my "server". Keep up the good work!
Hi @Brandawg93. Just wanted to confirm that removing the need to encode makes a big difference!
FYI,I have a Wyze Cam v2 as well (can’t go wrong for $25 compared to the $200 Nest Cam), and used the Camera FFMPEG plugin to integrate it into my HOOBS setup. (https://github.com/KhaosT/homebridge-camera-ffmpeg).
When I changed the codec from h264_omx to copy, obviating the need from the Raspberry Pi from processing the stream and just “copying” it thru, the difference was night and day.
The camera stream was not only more stable and more responsive. The delay and the slow-mo effect you currently see with this plugin also disappeared.
Hopefully, your experiments and reverse-engineering efforts to remove the encoding reqt pays off!
Interesting. I'm assuming that the Wyze Cam has a rtsp source endpoint that the plugin can simply pass through to Homekit. With that incoming stream, ffmpeg understands that it's h264 coming in. With Nest, the camera feed has no rtsp endpoint but rather a websocket of raw data. FFmpeg has no idea what it is, just that it needs to be encoded to be understood. Hopefully, I'll make a break-through on it one day.
Well...I figured it out! I've removed the need to transcode! The problem was that Nest does not send timestamps with the video stream, so I had to set them manually. New release coming soon!
Yep. Wyze cam has alternative beta firmware with RTSP support.
https://support.wyzecam.com/hc/en-us/articles/360026245231-Wyze-Cam-RTSP
And so glad you figured it out! Looking forward to the new release!
Well...I figured it out! I've removed the need to transcode! The problem was that Nest does not send timestamps with the video stream, so I had to set them manually. New release coming soon!
Hello, I just installed this and great work! I noticed the lag too. With the new release, will this fix that and once a new release comes out, do I have to do anything? I'm new to terminal on Mac and home bridge.
The new releases do not fix the latency. You can try to use the "copy" codec to reduce latency, but it may make the stream choppy. If you are new to shell commands, I would suggest using homebridge-config.
You can view the releases and the changes from each on the release page.
The stream is only about a 2-3 second delay which is fine, but it seems choppy. I have not made the "copy" suggestion yet as you said that COULD make it choppy. Any idea how to make the stream smooth? I have it running on homebridge / iMac terminal