Cannot Screen Share between iOS Simulator instances

Question

Cannot Screen Share between iOS Simulator instances

DnTest34324532 opened this issue a year ago · 2 comments

Thanks for the great framework/libraries/service! Everything else so far seems to work fine. I can see the other participant enter and exit the room, go on mute, etc.

However, when screensharing, the publish times out. This is with the latest version of this repo with the iOS Debug build. It looks like maybe something went wrong on the dimensions? Any insights into this? Thanks!

Server logs:

2023-04-30T21:52:11.004-0400    ERROR   livekit supervisor/participant_supervisor.go:142        supervisor error on publication {"room": "my-first-room", "roomID": "RM_TTo3SyNqzFkn", "participant": "user2", "pID": "PA_DoTUjnRMVgp4", "remote": false, "trackID": "TR_VSXd3GvwCmb58g", "error": "publish time out"}
github.com/livekit/livekit-server/pkg/rtc/supervisor.(*ParticipantSupervisor).checkPublications
        github.com/livekit/livekit-server/pkg/rtc/supervisor/participant_supervisor.go:142
github.com/livekit/livekit-server/pkg/rtc/supervisor.(*ParticipantSupervisor).checkState
        github.com/livekit/livekit-server/pkg/rtc/supervisor/participant_supervisor.go:131

Client logs (the one sharing the screen):

2023-04-30T21:51:36-0400 debug LiveKitSDK : [] MulticastDelegate<ParticipantDelegate>.notify(label:_:) [notify] participant.didUpdate connectionQuality: ConnectionQuality(rawValue: 3)
2023-04-30T21:51:36-0400 debug LiveKitSDK : [] MulticastDelegate<RoomDelegateObjC>.notify(label:_:) [notify] room.didUpdate connectionQuality: ConnectionQuality(rawValue: 3)
2023-04-30T21:51:40-0400 info LiveKitSDK : [] LocalParticipant.publish(track:publishOptions:) [publish] LocalVideoTrack(sid: nil, name: screen_share, source: Source(rawValue: 3)) options: nil...
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] MulticastDelegate<VideoCapturerDelegate>.notify(label:_:) [notify] capturer.didUpdate state: CapturerState(rawValue: 1)
2023-04-30T21:51:40-0400 debug LiveKitSDK : [LiveKit] open socket connection
2023-04-30T21:51:40-0400 debug LiveKitSDK : [LiveKit] failure: fd path is too long
2023-04-30T21:51:40-0400 debug LiveKitSDK : [LiveKit] failed setting up address
2023-04-30T21:51:40-0400 debug LiveKitSDK : [LiveKit] unscheduleStreams
2023-04-30T21:51:40-0400 debug LiveKitSDK : [LiveKit] closing server socket
2023-04-30T21:51:40-0400 debug LiveKitSDK : [LiveKit] unscheduleStreams
2023-04-30T21:51:40-0400 debug LiveKitSDK : [LiveKit] closing server socket
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] LocalParticipant.publish(track:publishOptions:) [publish] waiting for dimensions to resolve...
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] BroadcastScreenCapturer.dimensions [publish] nil -> Optional(<LiveKit.Dimensions: 0x6000027b0e30>)
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] BroadcastScreenCapturer.dimensions [publish] dimensions: Optional(<LiveKit.Dimensions: 0x6000027b0e30>)
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Completer<Dimensions>.wait(on:_:throw:) [publish] wait created...
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Completer<Dimensions>.set(value:) [publish] fulfill <LiveKit.Dimensions: 0x6000027b0e30> func: Optional((Function))...
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.sendAddTrack(cid:name:type:source:_:) 
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] LocalParticipant.publish(track:publishOptions:) [publish] computing encode settings with dimensions: <LiveKit.Dimensions: 0x6000027b0e30>...
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Utils.computeEncodings(dimensions:publishOptions:isScreenShare:) Using presets: [<LiveKit.VideoParameters: 0x600002527fc0>], count: 1 isScreenShare: true
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] LocalParticipant.publish(track:publishOptions:) [publish] using encodings: [RTCRtpEncodingParameters(rid: q, active: true, scaleResolutionDownBy: Optional(2), maxBitrateBps: Optional(150000), maxFramerate: Optional(3)), RTCRtpEncodingParameters(rid: h, active: true, scaleResolutionDownBy: Optional(1), maxBitrateBps: Optional(1500000), maxFramerate: Optional(15))]
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] LocalParticipant.publish(track:publishOptions:) [publish] using layers: VideoLayer(quality: low, dimensions: 443x960, bitrate: 150000), VideoLayer(quality: medium, dimensions: 886x1920, bitrate: 1500000)
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] LocalParticipant.publish(track:publishOptions:) [publish] requesting add track to server with AddTrackRequest(cid: 55C24B38-F344-4E23-AFCA-10DC0FF405D2, name: screen_share, type: video, source: screenShare, width: 886, height: 1920, muted: false)...
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Completer<Livekit_TrackInfo>.wait(on:_:throw:) [publish] wait created...
2023-04-30T21:51:40-0400 warning LiveKitSDK : [] SignalClient.notify(requiresHandle:function:line:label:_:) notify() was not handled by the delegate, called from init() line 77
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.onSignalResponse(_:) [publish] resolving completer for cid: 55C24B38-F344-4E23-AFCA-10DC0FF405D2
2023-04-30T21:51:40-0400 warning LiveKitSDK : [] SignalClient.notify(requiresHandle:function:line:label:_:) notify() was not handled by the delegate, called from onSignalResponse(_:) line 322
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.completeCompleter(forAddTrackRequest:trackInfo:) [publish] found the completer resolving...
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Completer<Livekit_TrackInfo>.set(value:) [publish] fulfill TrackInfo(sid: TR_VSXd3GvwCmb58g, name: screen_share, type: video, source: screenShare, width: 886, height: 1920, muted: false) func: Optional((Function))...
2023-04-30T21:51:40-0400 warning LiveKitSDK : [] SignalClient.notify(requiresHandle:function:line:label:_:) notify() was not handled by the delegate, called from init() line 77
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] LocalParticipant.publish(track:publishOptions:) [publish] server responded trackInfo: TrackInfo(sid: TR_VSXd3GvwCmb58g, name: screen_share, type: video, source: screenShare, width: 886, height: 1920, muted: false)
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] LocalParticipant.publish(track:publishOptions:) [publish] added transceiver: TrackInfo(sid: TR_VSXd3GvwCmb58g, name: screen_share, type: video, source: screenShare, width: 886, height: 1920, muted: false)...
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] LocalParticipant.publish(track:publishOptions:) [publish] set degradationPreference to .maintainResolution
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.publisherShouldNegotiate() 
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] LocalTrackPublication.set(track:) nil -> Optional(LocalVideoTrack(sid: nil, name: screen_share, source: Source(rawValue: 3)))
2023-04-30T21:51:40-0400 info LiveKitSDK : [] LocalParticipant.publish(track:publishOptions:) [publish] success LocalTrackPublication(sid: TR_VSXd3GvwCmb58g, kind: Kind(rawValue: 1), source: Source(rawValue: 3))
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] MulticastDelegate<RoomDelegateObjC>.notify(label:_:) [notify] localParticipant.didPublish LocalTrackPublication(sid: TR_VSXd3GvwCmb58g, kind: Kind(rawValue: 1), source: Source(rawValue: 3))
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] MulticastDelegate<ParticipantDelegate>.notify(label:_:) [notify] localParticipant.didPublish LocalTrackPublication(sid: TR_VSXd3GvwCmb58g, kind: Kind(rawValue: 1), source: Source(rawValue: 3))
2023-04-30T21:51:40-0400 warning LiveKitSDK : [] VideoView.performLayout() dimensions are nil, cannot layout without dimensions, track: LocalVideoTrack(sid: TR_VSXd3GvwCmb58g, name: screen_share, source: Source(rawValue: 3))
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] VideoView.createNativeRendererView() Using RTCEAGLVideoView for VideoView's Renderer
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] MulticastDelegate<VideoViewDelegate>.notify(label:_:) [notify] videoView.didUpdate viewSize: (383.0, 347.66666666666663)
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] MulticastDelegate<TrackDelegate>.notify(label:_:) [notify] track.didAttach videoView: <LiveKit.VideoView: 0x7f936df4ca40; frame = (0 0; 383 347.667); clipsToBounds = YES; autoresize = W+H; layer = <CALayer: 0x600002518e60>>
2023-04-30T21:51:40-0400 warning LiveKitSDK : [] VideoView.performLayout() dimensions are nil, cannot layout without dimensions, track: LocalVideoTrack(sid: TR_VSXd3GvwCmb58g, name: screen_share, source: Source(rawValue: 3))
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.configureTransports(joinResponse:) publisher onOffer v=0
o=- 3825336985156486459 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=extmap-allow-mixed
a=msid-semantic: WMS
m=video 9 UDP/TLS/RTP/SAVPF 96 97 100 101 35 36 125 124 127
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:Ftf6
a=ice-pwd:+GwALWVYXJR8W2Gn827tqpbO
a=ice-options:trickle renomination
a=fingerprint:sha-256 3A:3A:1B:68:40:B4:E1:4B:05:AC:F8:96:03:15:E7:61:5F:F1:B9:43:5F:AA:21:F7:3B:6D:B9:89:69:6F:4F:08
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 urn:3gpp:video-orientation
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=extmap:12 https://aomediacodec.github.io/av1-rtp-spec/#dependency-descriptor-rtp-header-extension
a=sendonly
a=msid:- 55C24B38-F344-4E23-AFCA-10DC0FF405D2
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 H264/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e032
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:35 AV1/90000
a=rtcp-fb:35 goog-remb
a=rtcp-fb:35 transport-cc
a=rtcp-fb:35 ccm fir
a=rtcp-fb:35 nack
a=rtcp-fb:35 nack pli
a=rtpmap:36 rtx/90000
a=fmtp:36 apt=35
a=rtpmap:125 red/90000
a=rtpmap:124 rtx/90000
a=fmtp:124 apt=125
a=rtpmap:127 ulpfec/90000
a=rid:q send
a=rid:h send
a=simulcast:send q;h
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=ice-ufrag:Ftf6
a=ice-pwd:+GwALWVYXJR8W2Gn827tqpbO
a=ice-options:trickle renomination
a=fingerprint:sha-256 3A:3A:1B:68:40:B4:E1:4B:05:AC:F8:96:03:15:E7:61:5F:F1:B9:43:5F:AA:21:F7:3B:6D:B9:89:69:6F:4F:08
a=setup:actpass
a=mid:1
a=sctp-port:5000
a=max-message-size:262144

2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Transport.peerConnection(_:didGenerate:) Did generate ice candidates RTC_OBJC_TYPE(RTCIceCandidate):
0
0
candidate:1288984508 1 udp 2122260223 192.168.124.144 53689 typ host generation 0 ufrag NQQK network-id 1 network-cost 10
 for subscriber
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.sendOffer(offer:) 
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Transport.peerConnection(_:didGenerate:) Did generate ice candidates RTC_OBJC_TYPE(RTCIceCandidate):
0
0
candidate:1288984508 1 udp 2122260223 192.168.124.144 56451 typ host generation 0 ufrag NQQK network-id 1 network-cost 10
 for subscriber
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.transport(_:didGenerate:) didGenerate iceCandidate
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.sendCandidate(candidate:target:) target: subscriber
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Transport.peerConnection(_:didGenerate:) Did generate ice candidates RTC_OBJC_TYPE(RTCIceCandidate):
0
0
candidate:1288984508 1 udp 2122260223 192.168.124.144 51795 typ host generation 0 ufrag Ftf6 network-id 1 network-cost 10
 for publisher
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.transport(_:didGenerate:) didGenerate iceCandidate
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.transport(_:didGenerate:) didGenerate iceCandidate
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Transport.peerConnection(_:didGenerate:) Did generate ice candidates RTC_OBJC_TYPE(RTCIceCandidate):
1
1
candidate:1288984508 1 udp 2122260223 192.168.124.144 58188 typ host generation 0 ufrag Ftf6 network-id 1 network-cost 10
 for publisher
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.sendCandidate(candidate:target:) target: publisher
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.sendCandidate(candidate:target:) target: subscriber
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.transport(_:didGenerate:) didGenerate iceCandidate
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.sendCandidate(candidate:target:) target: publisher
2023-04-30T21:51:40-0400 warning LiveKitSDK : [] SignalClient.notify(requiresHandle:function:line:label:_:) notify() was not handled by the delegate, called from onSignalResponse(_:) line 302
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Room.signalClient(_:didUpdate:) participants: [LiveKit.Livekit_ParticipantInfo:
sid: "PA_DoTUjnRMVgp4"
identity: "user2"
state: ACTIVE
joined_at: 1682905891
name: "user2"
version: 10
permission {
  can_subscribe: true
  can_publish: true
  can_publish_data: true
}
is_publisher: true
]
2023-04-30T21:51:40-0400 warning LiveKitSDK : [] SignalClient.notify(requiresHandle:function:line:label:_:) notify() was not handled by the delegate, called from onSignalResponse(_:) line 315
2023-04-30T21:51:40-0400 warning LiveKitSDK : [] SignalClient.notify(requiresHandle:function:line:label:_:) notify() was not handled by the delegate, called from onSignalResponse(_:) line 312
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Transport.peerConnection(_:didChange:) did update state connecting for publisher
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.transport(_:didUpdate:) target: publisher, state: connecting
2023-04-30T21:51:40-0400 warning LiveKitSDK : [] SignalClient.notify(requiresHandle:function:line:label:_:) notify() was not handled by the delegate, called from onSignalResponse(_:) line 312
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Transport.peerConnection(_:didGenerate:) Did generate ice candidates RTC_OBJC_TYPE(RTCIceCandidate):
0
0
candidate:622729341 1 udp 1686052607 74.64.230.117 56451 typ srflx raddr 192.168.124.144 rport 56451 generation 0 ufrag NQQK network-id 1 network-cost 10
stun:stun.l.google.com:19302 for subscriber
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.transport(_:didGenerate:) didGenerate iceCandidate
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.sendCandidate(candidate:target:) target: subscriber
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Transport.peerConnection(_:didGenerate:) Did generate ice candidates RTC_OBJC_TYPE(RTCIceCandidate):
0
0
candidate:622729341 1 udp 1686052607 74.64.230.117 51795 typ srflx raddr 192.168.124.144 rport 51795 generation 0 ufrag Ftf6 network-id 1 network-cost 10
stun:stun.l.google.com:19302 for publisher
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.transport(_:didGenerate:) didGenerate iceCandidate
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.sendCandidate(candidate:target:) target: publisher
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Transport.peerConnection(_:didGenerate:) Did generate ice candidates RTC_OBJC_TYPE(RTCIceCandidate):
0
0
candidate:622729341 1 udp 1686052607 74.64.230.117 53689 typ srflx raddr 192.168.124.144 rport 53689 generation 0 ufrag NQQK network-id 1 network-cost 10
stun:stun.l.google.com:19302 for subscriber
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.transport(_:didGenerate:) didGenerate iceCandidate
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.sendCandidate(candidate:target:) target: subscriber
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Transport.peerConnection(_:didGenerate:) Did generate ice candidates RTC_OBJC_TYPE(RTCIceCandidate):
0
0
candidate:39299916 1 tcp 1518280447 192.168.124.144 61479 typ host tcptype passive generation 0 ufrag Ftf6 network-id 1 network-cost 10
 for publisher
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.transport(_:didGenerate:) didGenerate iceCandidate
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] SignalClient.sendCandidate(candidate:target:) target: publisher
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Transport.peerConnection(_:didChange:) did update state connected for publisher
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Engine.transport(_:didUpdate:) target: publisher, state: connected
2023-04-30T21:51:40-0400 debug LiveKitSDK : [] Completer<()>.set(value:) [publish] fulfill () func: nil...
2023-04-30T21:51:41-0400 debug LiveKitSDK : [] SignalClient.sendPing() ping/pong: sending...
``

Answer 1 · 2023-05-01T05:35:52.000Z

I actually just saw the iOS Simulator Limitations. Am I understanding correctly that "Publishing the camera track is not supported by iOS Simulator" means screen share won't work in the simulator?

Answer 2 · 2023-05-03T07:53:08.000Z

@DnTest34324532 That seems correct. screen sharing has not been tested with iOS Simulator.