How can I decrease the latency below the 2second to 400ms or 500ms
vishaldybot opened this issue · 17 comments
Hi @razorRun, first of all, I want to thank you for your efforts and hard work in creating this library; it's very helpful.
Actually, my problem involves sending a video stream from Gstreamer using rtspserver.py and receiving the same in your react-native-vlc-media-player library. I was able to receive the stream successfully, but the latency is too high, around 2 seconds, which is the same as what I was receiving on the VLC desktop. I made some changes, such as {:network-caching=0 :0latency}, and I was able to reduce the latency to 400ms. However, when I attempt to do the same in this library, the latency doesn't decrease.
Could you please help me with this?
Thank you in advance!
rtspserver,py
import gi
gi.require_version('Gst', '1.0')
gi.require_version('GstRtspServer', '1.0')
from gi.repository import Gst, GstRtspServer, GObject
Gst.init(None)
mainloop = GObject.MainLoop()
Create RTSP server
server = GstRtspServer.RTSPServer.new()
server.set_address("192.168.1.72") # Set server's IP
server.set_service("8554") # Set the port where to serve
mounts = server.get_mount_points()
factory = GstRtspServer.RTSPMediaFactory.new()
Set pipeline
#factory.set_launch('( v4l2src do-timestamp=true io-mode=mmap device=/dev/video0 ! videoconvert ! video/x-raw,width=640,height=480,framerate=30/1 ! x264enc tune=zerolatency speed-preset=faster ! rtph264pay name=pay0 pt=96 )')
factory.set_launch('( v4l2src do-timestamp=true io-mode=mmap device=/dev/video0 ! videoconvert ! video/x-raw,width=640,height=480,framerate=30/1 ! x264enc tune=zerolatency speed-preset=faster ! rtph264pay name=pay0 pt=96 )')
#factory.set_launch('( v4l2src do-timestamp=true io-mode=mmap device=/dev/video0 ! videoconvert ! video/x-raw,width=640,height=480,framerate=30/1 ! omxh264enc control-rate=variable target-bitrate=1000000 ! video/x-h264,profile=high ! h264parse ! rtph264pay name=pay0 pt=96 )')
mounts.add_factory("/test", factory)
server.attach(None)
print(f"RTSP server ready at rtsp://192.168.1.72:8554/test")
mainloop.run()
import React from 'react';
import { VLCPlayer } from 'react-native-vlc-media-player';
react code:-
function Camera() {
return (
<VLCPlayer
style={{ height: 480, width: 640 }}
videoAspectRatio="16:9"
source={{
uri: 'rtsp://192.168.70.50:8554/test',
autoplay: true,
// initOptions: [/"--no-audio", "--no-stats", "--rtsp-tcp",/ "--0latency", "--no-hw-dec"] // I have commented it out as it was not working
}}
/>
);
}
export default Camera;
Hey @vishaldybot
Can you please try on the test app.. from the memory we managed to get the latency to around 500-1000ms over the internet (Around 400s local) when i was working on this. However this will depend on the stream quality as well. Have a play reducing that. Also hermes helped a bit as well
Hi @razorRun
As you mentioned, I tried my best to achieve the desired outcome, but I'm unable to reduce the latency below 2 seconds regardless of my efforts. The pipeline I'm using to transmit the video feed has a latency of only 10-20ms. However, when it comes to this library, I'm struggling to achieve the same results. I might be overlooking something.
I'd appreciate your assistance. React Native doesn't have another library offering this functionality. I've implemented all the other features but this one has me stuck.
rtspserver.py:
import gi
gi.require_version('Gst', '1.0')
gi.require_version('GstRtspServer', '1.0')
from gi.repository import Gst, GstRtspServer, GObject
Gst.init(None)
mainloop = GObject.MainLoop()
server = GstRtspServer.RTSPServer.new()
server.set_address("192.168.22.227") # Set server's IP
server.set_service("8554") # Set the port to serve
mounts = server.get_mount_points()
factory = GstRtspServer.RTSPMediaFactory.new()
factory.set_launch('( v4l2src do-timestamp=true io-mode=mmap device=/dev/video0 ! videoconvert ! video/x-raw,width=640,height=480,framerate=30/1 ! x264enc tune=zerolatency speed-preset=ultrafast key-int-max=30 bframes=0 threads=1 ! rtph264pay name=pay0 pt=96 )')
mounts.add_factory("/test", factory)
server.attach(None)
print(f"RTSP server ready at rtsp://192.168.22.227:8554/test")
mainloop.run()
app.tsx:
import React from 'react';
import { VLCPlayer } from 'react-native-vlc-media-player';
import { View } from 'react-native';
function App() {
return (
<View style={{ flex: 1 }}>
<VLCPlayer
autoplay={true}
autoAspectRatio={true}
style={{
height: 480, width: 640
}}
source={{
initType: 2,
hwDecoderEnabled: 1,
hwDecoderForced: 1,
uri: 'rtsp://192.168.22.227:8554/test',
initOptions: [
"--rtsp-tcp",
"--network-caching=150",
"--rtsp-caching=150",
"--no-stats",
"--tcp-caching=150",
"--realrtsp-caching=150",
],
}}
isLive={true}
autoReloadLive={true}
onPlaying={(e) => {
console.log("aaa", e);
}}
/>
);
}
export default App;
it`s a request that please give it a try and see whether it is working or not and if not please help me out with this as the react native does not have any other way to do the same.
@vishaldybot any update on this?
Yes, The issue has been resolved
Yes, The issue has been resolved
How did you get it working with low latency?
By modifying the
--network-caching=0',
'--rtsp-caching=0',
If this doesn`t work please do let me know I will try to resolve the issue
By modifying the --network-caching=0', '--rtsp-caching=0',
If this doesn`t work please do let me know I will try to resolve the issue
I have tried with this already and it didn't work, latency was still upwards of 1000ms
also you need to disable hardware decoder to 0
hwDecoderEnabled: 0,
hwDecoderForced: 0,
Hi, i'm having similar issue, my VLC player on mac plays the rtsp stream fine. But on the app, there is a huge delay, it shows up but only like 1 frame every minute. I followed the example App, any ideas?
May I know the motive that what do you want to do ??
so that I would be able to suggest some other alternatives too else if you are interested in VLC than I will try to help you out with this only
@Wolfleader101 Please share your standalone code I will debug it and resolve and share
but I will do in android only
Hi @vishaldybot i have a Linux computer as the rtsp server with IP camera. Streaming from my Mac VLC player is ok with slight delay. But using this package on react native there's a lot of delay like 1 frame per minute. Playing mp4 uri is fine on RN. I used the sample test code and still same problem. Tried the different options above as well.
ios17. Was looking at Flutter library seems people are having the same problem with iOS 17 solid-software/flutter_vlc_player#444 - something to do with com.apple.developer.networking.multicast
I'm going to try Android later
Edit: I tested on Android and it's much better, with the 2sec delay or so, any solution for iOS?
Just circling back - after adding the Multicast Networking to xcode, works now with the bearable sub 1 second delay
how latency you are expecting??
@jwee1369 Is VLC working normally for you on iOS version 17?
By modifying the --network-caching=0', '--rtsp-caching=0',
If this doesn`t work please do let me know I will try to resolve the issue
@razorRun
I am doing live streaming using rtmp server, but streaming video is taking time to load and frequently showing white screen before loading.
and also getting stucked while playing, this issue is mainly related to iOS.
bitrate is 1500 kbps.
<VLCPlayer
ref={playerRef}
source={{
uri: liveStreamingUrl ? liveStreamingUrl : "",
}}
style={styles.videoPlayer}
videoAspectRatio={isFullscreen ? "16:9" : "3:2"}
resizeMode="contain"
paused={!isPlaying}
onError={(e) => console.log("onError", e)}
onPlaying={onPlaying}
onPaused={onPaused}
mediaOptions={{
":network-caching": 150,
":live-caching": 300,
":file-caching": 0,
":network-caching": 150,
":clock-jitter": 0,
":clock-synchro": 0,
}}
/>
