How to reduce latency when switching videos?

[Naozumi520]

As title, how to switch the video smoother? Currently, I use the streamLivestreamVideo method, and I hope to have the similar effect like changing windows on discord. Using streamLivestreamVideo will have few seconds of the delay and grey screen.

[dank074]

Unfortunately, since we rely on a child process (ffmpeg) to re-encode the video, we are bound to ffmpeg's latency. There's several flags that you can use to reduce the latency (https://stackoverflow.com/questions/16658873/how-to-minimize-the-delay-in-a-live-streaming-with-ffmpeg), but the current streamLivestreamVideo method is not very flexible and all the flags are hardcoded (maybe this can be improved). That means you will have to write your own streaming method to use custom ffmpeg flags.

This project is also open to PRs so if you experiment with the flags and find a good combination it would be cool if you would contribute those.

[dank074]

Wrote a snippet at someone's request that mimicked "changing windows" because we change the source video, and I didn't experience a long delay like you described. This is the code I tried:

import { Client } from "discord.js-selfbot-v13";
import { command, streamLivestreamVideo, MediaUdp, setStreamOpts, Streamer } from "@dank074/discord-video-stream";
import config from "./config.json";

const streamer = new Streamer(new Client());

setStreamOpts({
    width: config.streamOpts.width, 
    height: config.streamOpts.height, 
    fps: config.streamOpts.fps, 
    bitrateKbps: config.streamOpts.bitrateKbps,
    maxBitrateKbps: config.streamOpts.maxBitrateKbps, 
    hardware_acceleration: config.streamOpts.hardware_acceleration,
    video_codec: config.streamOpts.videoCodec === 'H264' ? 'H264' : 'VP8'
})

// ready event
streamer.client.on("ready", (client) => {
    console.log(`--- ${client.user.tag} is ready ---`);
});

// message event
streamer.client.on("messageCreate", async (msg) => {
    if (msg.author.bot) return;

    if (!config.acceptedAuthors.includes(msg.author.id)) return;

    if (!msg.content) return;

    if(msg.content.startsWith('$join')) {

        const channel = msg.author.voice.channel;

        if(!channel) {
            console.log('you must be in a voice channel first');
            return;
        }

        console.log(`Attempting to join voice channel ${msg.guildId}/${channel.id}`);
        await streamer.joinVoice(msg.guildId, channel.id);
    } else if (msg.content.startsWith(`$play-live`)) {
        const args = parseArgs(msg.content)
        if (!args) return;

        if(!streamer.voiceConnection) {
            console.log('bot must be in a voice channel')
            return;
        }

        // lets stop the current ffmpeg instance first
        command?.kill('SIGINT');

        // lets give the process time to close before we spawn a new one
        await new Promise<void>((resolve, reject) => {
            setTimeout(() => {
                resolve();
            }, 500);
        });

        if(!streamer.voiceConnection.streamConnection) {
            // no current stream has started, so lets start one
            const udpConn = await streamer.createStream();

            udpConn.mediaConnection.setSpeaking(true);
            udpConn.mediaConnection.setVideoStatus(true);
        }
        
        playVideo(args.url, streamer.voiceConnection.streamConnection.udp);
        return;
    } else if (msg.content.startsWith("$disconnect")) {
        command?.kill("SIGINT");

        streamer.leaveVoice();
    } else if(msg.content.startsWith("$stop-stream")) {
        command?.kill('SIGINT');

        const stream = streamer.voiceConnection?.streamConnection;

        if(!stream) return;

        stream.setSpeaking(false);
        stream.setVideoStatus(false);

        streamer.stopStream();
    }
});

// login
streamer.client.login(config.token);

async function playVideo(video: string, udpConn: MediaUdp) {
    console.log("Started playing video");
    try {
        const res = await streamLivestreamVideo(video, udpConn);

        console.log("Finished playing video " + res);
    } catch (e) {
        console.log(e);
    }
}

function parseArgs(message: string): Args | undefined {
    const args = message.split(" ");
    if (args.length < 2) return;

    const url = args[1];

    return { url }
}

type Args = {
    url: string;
}

[Hidend]

How can i know if I am screensharing or not? I don't get have screenShareConn somehow

[dank074]

How can i know if I am screensharing or not? I don't get have screenShareConn somehow

Will update snippet to work on latest version later tonight

Edit: Updated

[Pim112]

My command is undefined. Because i can't kill ffmpeg processes the module becomes unusable after 1 video.