Pause / resume stream

[zxcvk6]

how can i pause the stream ?
further how to get video player features (speed, volume, ...)

[Mrx-dev]

use command.kill('SIGSTOP'); for Pause and for resume use command.kill('SIGCONT');

[DevTobias]

So I noticed that stopping the ffmpeg process desyncs the audio and lags out the video a bit. Insteadt, wouldn't it be a solution to let ffmpeg run and buffer the video? For example in the VideoStream/AudioStream file, you could do something like this:

  _write(frame: Buffer, _: BufferEncoding, callback: (error?: Error | null) => void) {
    this.count += 1;
    if (!this.startTime) this.startTime = Date.now();

    this._frameBuffer.push(frame);

    if (!this._paused) this.udb()?.sendVideoFrame(this._frameBuffer.shift() ?? frame);

    const next = (this.count + 1) * this.sleepTime - (Date.now() - this.startTime);
    setTimeout(() => callback(), next);
  }

To make this more robust, you must await all remaining frames in the buffer, before the ffmpeg command calls it end event. Also I see a potential problem for larger video files if paused to long, this could fill up the RAM. So I gues you could also add a threshold, to only stop the ffmpeg process after a certain buffer size. This should also solve the desync problem.

[dank074]

So I noticed that stopping the ffmpeg process desyncs the audio and lags out the video a bit. Insteadt, wouldn't it be a solution to let ffmpeg run and buffer the video? For example in the VideoStream/AudioStream file, you could do something like this:

  _write(frame: Buffer, _: BufferEncoding, callback: (error?: Error | null) => void) {
    this.count += 1;
    if (!this.startTime) this.startTime = Date.now();

    this._frameBuffer.push(frame);

    if (!this._paused) this.udb()?.sendVideoFrame(this._frameBuffer.shift() ?? frame);

    const next = (this.count + 1) * this.sleepTime - (Date.now() - this.startTime);
    setTimeout(() => callback(), next);
  }

To make this more robust, you must await all remaining frames in the buffer, before the ffmpeg command calls it end event. Also I see a potential problem for larger video files if paused to long, this could fill up the RAM. So I gues you could also add a threshold, to only stop the ffmpeg process after a certain buffer size. This should also solve the desync problem.

That sounds like a great solution. You can open if you wish or I can see if I can implement it once I get some free time

[yannikwirbs]

Hey hey I found a way to be able to pause and resume the stream.
I basically stopped the stream to send the callback for the time being paused.
The stream doesn't send a new frame so long it didnt receive the callback.
Here are both ts classes:

VideoStream

import { MediaUdp } from "@dank074/discord-video-stream";
import stream from "stream";

export default class VideoStream extends stream.Writable {
  udp: MediaUdp;
  count: number;
  sleepTime: number;
  startTime: any;
  paused: boolean = false;
  buffer: Buffer[] = [];

  constructor(udp: any, fps = 30) {
    super();
    this.udp = udp;
    this.count = 0;
    this.sleepTime = 1000 / fps;
  }

  setSleepTime(time: number) {
    this.sleepTime = time;
  }

  async _write(frame: any, encoding: BufferEncoding, callback: (error?: Error | null) => void) {
    if (!this.startTime) this.startTime = performance.now();

    do {
      this.count++;
      this.udp.sendVideoFrame(frame);
      const next = (this.count + 1) * this.sleepTime - (performance.now() - this.startTime);
      await delay(next);
    } while (this.paused);

    callback();
  }

  pause() {
    this.paused = !this.paused;
  }
}

function delay(ms: number) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

AudioStream

import { MediaUdp } from "@dank074/discord-video-stream";
import stream from "stream";

export default class AudioStream extends stream.Writable {
  paused: boolean = false;
  udp: MediaUdp;
  count: number;
  sleepTime: number;
  startTime: any;
  buffer: any[] = [];
  constructor(udp: MediaUdp) {
    super();
    this.udp = udp;
    this.count = 0;
    this.sleepTime = 20;
  }

  async _write(chunk: any, _: BufferEncoding, callback: (error?: Error | null) => void) {
    if (!this.startTime) this.startTime = performance.now();
    this.udp.sendAudioFrame(chunk);
    do {
      this.count++;
      const next = (this.count + 1) * this.sleepTime - (performance.now() - this.startTime);
      await delay(next);
    } while (this.paused);
    callback();
  }

  pause() {
    this.paused = !this.paused;
  }
}

function delay(ms: number) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

I haven't quite figured out how to send a still frame so the camera or stream is not loading.
Currently it is showing a graduly more colorfully distorted picture.
I didn't have time to test it thoroughly yet. So far there is no desync or other.
I havent figured out what happens to ffmpeg over a long time of being paused or if it might crash.

The other solution is not really working. I have tried it. The video gets played as long as the ffmpeg process sends new frames. After it is done the stream doesnt receive any new instructions and therefore doesnt send the rest of the buffered frames.
Hope that helps. @dank074