livekit/client-sdk-js

Publish tracks from any source

Closed this issue · 1 comments

Describe the problem

Does sdkjs support publishing audio data to the room and then playing back the audio? This audio may be a blob object. Room recording can record the content of this audio playback.
Can I choose not to use shared desktop?

Describe the proposed solution

First, I tested the blob audio object to be converted to MediaStreamTrack and then published tracks from any source. I found that the blob audio could not be converted to MediaStreamTrack and the tracks could not be sent successfully. In the future, I sent a MediaStreamTrack for test, but the room could not receive this MediaStreamTrack and play it as audio. Is there any solution

`//Publish tracks code
const blob = await this.fetchAudio();
console.log("blob", blob)
const audioUrl = URL.createObjectURL(blob);
console.log("audioUrl", audioUrl)
this.audioPlayer = audioUrl;
const audioSource = AudioMixerPlugin.createAudioSource({url: audioUrl})
console.log("audioSource", audioSource)
const destination = audioSource.destination
console.log("destination", destination)
const mediaStream = destination.stream;
console.log("mediaStream", mediaStream)
const audioTracks = mediaStream.getAudioTracks();
console.log("audioTracks", audioTracks)
const audioTrack = audioTracks[0];
console.log("audioTrack", audioTrack)
const pub = await this.room.localParticipant.publishTrack(audioTrack, {
name: 'mytrack',
simulcast: true,
source: 'microphone',
});

//Monitor code
this.room.on(LivekitClient.RoomEvent.TrackPublished, function (track) {
console.log("track",track)
const audioElement = new Audio();
console.log("track",track.track) //This prints as null
const stream = new MediaStream([track.track]);
audioElement.srcObject = stream;
audioElement.play().catch(error => {
console.error('Error playing audio track:', error);
});
})`

Alternatives considered

No response

Importance

nice to have

Additional Information

//Publish tracks code
image
image

//Monitor code
image
image

the sending part looks right, for receiving you'll want to listen to RoomEvent.TrackSubscribed and for ease of use simply call track.attach(myHTMLAudioElement)