blackuy/react-native-twilio-video-webrtc

In Ios Participant Video is not visible

Closed this issue · 15 comments

vishallovecode commented

Steps to reproduce

Participant_1 => In any another platform web(chrome or safari) app(ios and android)
Participant_2 => In Ios app

  1. Initiate a call from Participant_1
  2. Call will be connected to Participant_2
  3. Participant_2 is not able to see Participant_1 video in 80% times. Blank screen is Visible.

Expected behaviour

Participant_2 should be able to see the Participant_1 video as expected

Actual behaviour

Participant_2 is not able to see the Participant_1 Video 80% times 20% times Participant_2 can see the Participant_1 Video

Environment

  • Node.js version: 14.15.1
  • React Native version: 0.64.1
  • React Native platform + platform version: iOS in every version

react-native-twilio-video-webrtc => git+https://github.com/blackuy/react-native-twilio-video-webrtc.git#6ebd6b6"

Version: git+https://github.com/blackuy/react-native-twilio-video-webrtc.git#6ebd6b6"
@slycoder Please help here.

vishallovecode commented

There is some mistakes in our own code.

ch3tan03 commented

There is some mistakes in our own code.

can you please share your code? I'm also facing the same issue

vishallovecode commented

@ch3tan03 can you please share the code where you are implementing the logic for showing participant videos

ch3tan03 commented

@ch3tan03 can you please share the code where you are implementing the logic for showing participant videos

const _onParticipantAddedVideoTrack = ({ participant, track }) => {
console.log("onParticipantAddedVideoTrack: ", participant, track);

setVideoTracks(
  new Map([
    ...videoTracks,
    [
      track.trackSid,
      { participantSid: participant.sid, videoTrackSid: track.trackSid },
    ],
  ])
);

};

<View style={styles.remoteGrid}>
            {Array.from(videoTracks, ([trackSid, trackIdentifier]) => {
              console.log(videoTracks)
              console.log('1',trackSid, trackIdentifier)
              return (
                <TwilioVideoParticipantView
                  enabled={true}
                  style={styles.remoteVideo}
                  key={trackSid}
                  trackIdentifier={trackIdentifier}
                />
              );
            })}
          </View>

In my case onParticipantAddedVideoTrack never gets called when a participant joins. sound is audible but no video

vishallovecode commented

@ch3tan03 Can you please share me the code where you are invoking _onParticipantAddedVideoTrack function.
Also please share me the onParticipantAddedVideoTrack Functions implementation.

ch3tan03 commented

@lpuvishal PFA full twilio component

export const VideoContainer = (props) => {
const isFocused = useIsFocused();
const [isAudioEnabled, setIsAudioEnabled] = useState(true);
const [isVideoEnabled, setIsVideoEnabled] = useState(true);
const [status, setStatus] = useState("disconnected");
const [participants, setParticipants] = useState(new Map());
const [videoTracks, setVideoTracks] = useState(new Map());
const [token, setToken] = useState("");
const [callActive, setCallActive] = useState(false);
const twilioVideo = useRef();
const [currentContact, scurrentContact] = useState(props.route.params.currentContact)
const [myData, smyData] = useState(props.route.params.myData)
const [isAudioCall, sisAudioCall] = useState(false);
const [outgoing, soutgoing] = useState(props.route.params.outgoing)
const socket = useContext(SocketContext);

const handleInviteAccepted = useCallback((data) => {

console.log(data)
switch (data.type) {
  case 'offer':

    console.log('offer', data);
    break;

  case 'answer':
    //    let _connect;
    //        _connect = async () => await _onConnectButtonPress(token)
    // _connect();
    _onConnectButtonPress(token === '' ? props.route.params.token : token)
    InCallManager.stopRingback();

    console.log('Answer', data, props.route.params.token);
    break;

  case 'reject':
    _onEndButtonPress()


    console.log('reject', data);
    break;


  default:
    break;
}

}, []);

useEffect(() => {

//console.log('56sock',socket,myData)

socket.emit('vcroom', myData.member_id);

if (outgoing) {

  socket.emit('vcRequests', {
    receiver: currentContact.to_id,
    sender: myData.to_id,
    senderData: myData,
    receiverData: currentContact,
    type: 'offer',

  });
  InCallManager.start({ media: 'audio', ringback: '_DTMF_' });
}

socket.on("socketVCEvent", handleInviteAccepted);

LogBox.ignoreLogs(['Animated: `useNativeDriver`']);
BackHandler.addEventListener('hardwareBackPress', backPress)


return () => {
  //  socket.removeAllListeners("socketVCEvent");
  socket.off("socketVCEvent");
  setCallActive(false)
  InCallManager.stopRingback();
  InCallManager.stopRingtone();
  InCallManager.stop();
}

}, [socket, myData]);

const backPress = () => true;

useEffect(() => {
//const access_token = async () => await AsyncStorage.getItem('access_token');

//  return () => {
//   twilioVideo.current.disconnect();
// };
//
//let _connect;

if (props.route.params.token === 'no') {
  generate_twilio_access_token(props.route.params.access_token, props.route.params.currentContact.to_id).then(response => {
    setToken(response.data.access_token)

    //  _connect = async () => await _onConnectButtonPress(response.data.access_token)
    // _connect();

  }).catch(err => { let error = err; console.log(error) })
}

}, [isFocused]);

const _onConnectButtonPress = async (twilioToken) => {
console.log(twilioToken, '112')
if (twilioToken === "") return;
setCallActive(true)
if (Platform.OS === "android") {
await _requestAudioPermission();
await _requestCameraPermission();
}
console.log('52---->', props.route.params, twilioToken)
try {
twilioVideo?.current?.connect({
roomName: props.route.params.roomName,
accessToken: twilioToken,
dominantSpeakerEnabled: true,
enableAudio: true,
enableVideo: true,
enableRemoteAudio: true,
enableNetworkQualityReporting: true,
maintainVideoTrackInBackground: true,
enableH264Codec: false,
// preferredAudioCodecs: ['isac'],
// preferredVideoCodecs: ['V8', 'H264']

  });
} catch (error) {
  console.log(error);
}

setStatus("connecting");

if (!outgoing) {
  //console.log("155",currentContact.to_id,myData.to_id)
  socket.emit('vcRequests', {
    receiver: currentContact.to_id,
    sender: myData.to_id,
    type: 'answer',

  });
}

};

const _onEndButtonPress = () => {

try {
  twilioVideo?.current?.disconnect();
}
catch(err) {}

setStatus("disconnected")
setCallActive(false)
goToChatMessageScreen(props.route.params.currentContact);
socket.emit('vcRequests', {
  receiver: currentContact.to_id,
  sender: myData.to_id,
  type: 'reject',

});
InCallManager.stopRingback();
InCallManager.stopRingtone();
InCallManager.stop();

};

const _onMuteButtonPress = () => {
console.log(twilioVideo.current)
twilioVideo.current
.setLocalAudioEnabled(!isAudioEnabled)
.then((isEnabled) => setIsAudioEnabled(isEnabled));
};

const _onVideoButtonPress = () => {

twilioVideo.current
  .setLocalVideoEnabled(!isAudioEnabled)
  .then((isEnabled) => setIsAudioEnabled(isEnabled));

};

const _onFlipButtonPress = () => {
twilioVideo.current.flipCamera();
};

const _onRoomDidConnect = () => {
console.log('63', 'connected')
setStatus("connected");
};

const _onRoomDidDisconnect = ({ error }) => {

setStatus("disconnected");

};

const _onRoomDidFailToConnect = (error) => {
console.log("ERROR: ", error);

setStatus("disconnected");

};

const _onParticipantAddedVideoTrack = ({ participant, track }) => {
console.log("onParticipantAddedVideoTrack: ", participant, track);

setVideoTracks(
  new Map([
    ...videoTracks,
    [
      track.trackSid,
      { participantSid: participant.sid, videoTrackSid: track.trackSid },
    ],
  ])
);

};

const _onParticipantRemovedVideoTrack = ({ participant, track }) => {
console.log("onParticipantRemovedVideoTrack: ", participant, track);

const newVideoTracks = new Map(videoTracks);
newVideoTracks.delete(track.trackSid);

setVideoTracks(newVideoTracks);

};
const _onRoomParticipantDidConnect = ({ roomName, participant }) => {
console.log("_onRoomParticipantDidConnect", roomName, participant);
};
const _onLocalParticipantSupportedCodecs = ({ supportedCodecs }) => {
console.log("_onRoomParticipantDidConnect", supportedCodecs);
};
const _onNetworkLevelChanged = ({ participant, isLocalUser, quality }) => {
console.log("Participant", participant, "isLocalUser", isLocalUser, "quality", quality);
};

const _onDominantSpeakerDidChange = ({ roomName, roomSid, participant }) => {
console.log("onDominantSpeakerDidChange", roomName: ${roomName}, roomSid: ${roomSid}, "participant:", participant);
};

const _requestAudioPermission = () => {
return PermissionsAndroid.request(
PermissionsAndroid.PERMISSIONS.RECORD_AUDIO,
{
title: "Need permission to access microphone",
message:
"To run this demo we need permission to access your microphone",
buttonNegative: "Cancel",
buttonPositive: "OK",
}
);
};

const _requestCameraPermission = () => {
return PermissionsAndroid.request(PermissionsAndroid.PERMISSIONS.CAMERA, {
title: "Need permission to access camera",
message: "To run this demo we need permission to access your camera",
buttonNegative: "Cancel",
buttonPositive: "OK",
});
};

return (

<View style={{
height: deviceDimesions.Height,
width: deviceDimesions.width,
}}>
{/* {status === "disconnected" && (

Are you sure you want to start video call?
<TextInput
style={styles.input}
autoCapitalize="none"
value={token}
onChangeText={(text) => setToken(text)}
>
<Button

        title="Connect"
        style={styles.button}
        onPress={_onConnectButtonPress}
      ></Button>
    </View>
  )} */}

    {(status === "connected" || status === "connecting") && (
      <View style={styles.callContainer}>
        {status === "connected" && (
          <View style={styles.remoteGrid}>
            {Array.from(videoTracks, ([trackSid, trackIdentifier]) => {
              console.log(videoTracks)
              console.log('1',trackSid, trackIdentifier)
              return (
                <TwilioVideoParticipantView
                  enabled={true}
                  style={styles.remoteVideo}
                  key={trackSid}
                  trackIdentifier={trackIdentifier}
                />
              );
            })}
          </View>
        )}
        <TwilioVideoLocalView enabled={true} style={styles.localVideo} />
        <View style={styles.optionsContainer}>


          <TouchableOpacity onPress={_onEndButtonPress} style={{ elevation: 3, backgroundColor: '#f94144', height: 45, width: 45, alignItems: 'center', borderRadius: 40, marginLeft: 5, justifyContent: 'center' }}>
            <Icon size={20} name="phone" color='#ffffff' />
          </TouchableOpacity>

          <TouchableOpacity onPress={_onMuteButtonPress} style={{ elevation: 3, backgroundColor: '#ffffff', height: 45, width: 45, alignItems: 'center', borderRadius: 40, marginLeft: 20, justifyContent: 'center' }}>
            <Icon size={20} name="microphone-slash" color='black' />
          </TouchableOpacity>

          <TouchableOpacity onPress={_onVideoButtonPress} style={{ elevation: 3, backgroundColor: '#ffffff', height: 45, width: 45, alignItems: 'center', borderRadius: 40, marginLeft: 20, justifyContent: 'center' }}>
            <Icon size={20} name="camera" color='black' />
          </TouchableOpacity>

          <TouchableOpacity onPress={_onFlipButtonPress} style={{ elevation: 3, backgroundColor: '#ffffff', height: 45, width: 45, alignItems: 'center', borderRadius: 40, marginLeft: 20, justifyContent: 'center' }}>
            <Icon size={20} name="undo" color='black' />
          </TouchableOpacity>

        </View>

      </View>
    )}

    {/* <TwilioVideo
    ref={twilioVideo}
    onRoomDidConnect={()=>_onRoomDidConnect()}
    onRoomDidDisconnect={()=>_onRoomDidDisconnect()}
    onRoomDidFailToConnect={()=>_onRoomDidFailToConnect()}
    onParticipantAddedVideoTrack={()=>_onParticipantAddedVideoTrack()}
    onParticipantRemovedVideoTrack={()=>_onParticipantRemovedVideoTrack()}
    onNetworkQualityLevelsChanged={()=>_onNetworkLevelChanged()}
    onDominantSpeakerDidChange={()=>_onDominantSpeakerDidChange()}
  /> */}
    <TwilioVideo
      ref={twilioVideo}
      onRoomDidConnect={_onRoomDidConnect}
      onRoomDidDisconnect={_onRoomDidDisconnect}
      onRoomDidFailToConnect={_onRoomDidFailToConnect}
      onParticipantAddedVideoTrack={_onParticipantAddedVideoTrack}
      onParticipantRemovedVideoTrack={_onParticipantRemovedVideoTrack}
      onRoomParticipantDidConnect={_onRoomParticipantDidConnect}
      onLocalParticipantSupportedCodecs={_onLocalParticipantSupportedCodecs}
      // onNetworkQualityLevelsChanged={_onNetworkLevelChanged}
      // onDominantSpeakerDidChange={_onDominantSpeakerDidChange}
    />
  </View>


  {!callActive && <View
    style={{
      backgroundColor: '#ffe9e9',

      justifyContent: 'center',
      alignItems: 'center',
      height: deviceDimesions.Height,
      width: deviceDimesions.width,
      position: 'absolute',
      elevation: 1,
      zIndex: 10
    }}>
    <ImageBackground source={ImagesPathVariable.heartbgoverlay} resizeMode="cover" style={{ width: deviceDimesions.width, height: deviceDimesions.Height, backgroundColor: '#ffe9e9' }}>

      <View pointerEvents="none" style={{ alignSelf: 'center', height: deviceDimesions.Height, marginTop: -deviceDimesions.Height * 0.3, width: deviceDimesions.width, justifyContent: 'center' }}>
        <PulseLoader borderColor="#ff751a"
          size={deviceDimesions.width * 0.5}
          avatarBackgroundColor="#ffffff"
          avatar={BaseURL.DemoURL + currentContact.profile_pic}
          pressInValue={0.6}
        />
      </View>

      <View style={{ marginTop: -deviceDimesions.Height * 0.3, alignItems: "center" }}>

        <Text style={{ fontSize: 22, marginBottom: 10, marginTop: 15, textTransform: "capitalize", fontFamily: 'Nunito-ExtraBold', }}>
          {currentContact.to_name}

        </Text>
        <Text style={{ fontSize: 18, marginBottom: 20 }}>
          {isAudioCall ? 'Audio' : 'Video'}

        </Text>

        {!outgoing ? <View style={{ flexDirection: 'row', zIndex: 1, marginVertical: 20 }}>

          <TouchableOpacity onPress={() => _onConnectButtonPress(token)} style={{ elevation: 3, backgroundColor: '#008000', height: 55, width: 55, alignItems: 'center', borderRadius: 40, marginLeft: 5, justifyContent: 'center', marginRight: 30 }}>
            <Icon size={25} name="phone" color='#ffffff' />
          </TouchableOpacity>
          <TouchableOpacity onPress={() => _onEndButtonPress} style={{ elevation: 3, backgroundColor: '#f94144', height: 55, width: 55, alignItems: 'center', borderRadius: 40, marginLeft: 5, justifyContent: 'center' }}>
            <Icon size={25} name="phone" color='#ffffff' style={{ transform: [{ rotate: '135deg' }] }} />
          </TouchableOpacity>

        </View> :
          <View style={{ flexDirection: 'row', zIndex: 1, marginVertical: 20 }}>

            <TouchableOpacity onPress={_onEndButtonPress} style={{ elevation: 3, backgroundColor: '#f94144', height: 55, width: 55, alignItems: 'center', borderRadius: 40, marginLeft: 5, justifyContent: 'center' }}>
              <Icon size={25} name="phone" color='#ffffff' style={{ transform: [{ rotate: '135deg' }] }} />
            </TouchableOpacity>

          </View>
        }
      </View>
    </ImageBackground>

  </View>}

</View>

);
};

vishallovecode commented 
      <View style={styles.remoteGrid}>
        {Array.from(videoTracks, ([trackSid, trackIdentifier]) => {
          console.log(videoTracks)
          console.log('1',trackSid, trackIdentifier)
          return (
            <TwilioVideoParticipantView
              enabled={true}
              style={styles.remoteVideo}
              key={trackSid}
              trackIdentifier={trackIdentifier}
            />
          );
        })}
      </View>
    )}

@ch3tan03 Are you getting trackSid ? above console.log code ?

ch3tan03 commented

onParticipantAddedVideoTrack never gets called when participants join. still, both users can hear each other but no video. whereas when I use ParticipantAddedDataTrack instead of onParticipantAddedVideoTrack I get full object { participant, track }.

vishallovecode commented

encodingParameters: {
enableH264Codec: true,
audioBitrate: 16,
videoBitrate: 1200
}
@ch3tan03 Please try using this above line instead of enableH264Codec: false
Are you facing this issue in all OS?

ch3tan03 commented

tried your solution, still same. sound is audible but no video for both users. for now, I have just checked in android.

vishallovecode commented

this.refs.twilioVideo.connect({
roomName: this.state.roomName,
accessToken: this.state.token,
enableVideo: true,
enableAudio: true,
enableRemoteAudio: true,
enableNetworkQualityReporting: true,
encodingParameters: {
enableH264Codec: true,
audioBitrate: 16,
videoBitrate: 1200
}
});
@ch3tan03 Please try this. If this is not working we can connect separately .

ch3tan03 commented

thanks for the prompt reply. tried the above solution as well still the same only audio. in my case below methods never get called.
onParticipantAddedVideoTrack={_onParticipantAddedVideoTrack}
onParticipantRemovedVideoTrack={_onParticipantRemovedVideoTrack}

I tried replacing these with onParticipantAddedDataTrack, onParticipantRemoveDataTrack but t gives a black screen.

ch3tan03 commented

Can you confirm your react native version, & below details buildToolsVersion = "29.0.2"
minSdkVersion = 21
compileSdkVersion = 29
targetSdkVersion = 29

vishallovecode commented

Hi @ch3tan03 Can we connect now ?

ch3tan03 commented

sure please connect I'm waiting in the meeting room. https://meet.google.com/qsn-jmxc-pnv