Using web worker to keep playing when window is not focused
Boscop opened this issue · 4 comments
Just an idea, maybe it's out of scope for this library, so feel free to close this :)
Both setInterval and requestAnimationFrame have issues for midi playback, they both don't get called often enough when the window is not focused. (requestAnimationFrame doesn't get called at all in chrome, and setInterval gets called only once per second or so.)
This is really annoying when switching from the midi-playing/looping browser window to the DAW where the live midi is being processed, it really disturbs the whole workflow :/
Web workers allow running a loop in a separate thread that runs even when the window isn't focused:
https://mortenson.coffee/blog/making-multi-track-tape-recorder-midi-javascript/
But setInterval is extremely unreliable, especially as users change tabs and reallocate resources. To make things more consistent, I created a new Web Worker just for the timing code so that it runs in its own thread. Web Workers also have more consistent performance when the page doesn't have focus.
In the above blog post, the author had good results with a web worker. But I would write the tick() function differently:
Instead of hoping that the time between calls stays constant, and trying to adjust the next timeout value based on deviation, I would use performance.now() to calculate the number of milliseconds since the last call, and then calculate how many ticks passed based on that, also storing the fractional tick remainder to add it to the number of ticks for the next frame, like this:
pub fn frame(&mut self, timestamp: DOMHighResTimeStamp) {
let start_timestamp = *self.m_start_timestamp.get_or_insert(timestamp);
if let Some(prev) = self.m_prev_timestamp && prev < timestamp {
let elapsed_since_prev_ms = timestamp - prev;
// Sent midi beat clock 24 times per beat
let cur_midi_beat_clock_frame = (self.playhead_tick as u64 * 24 / self.smf.ticks_per_beat as u64) as i32;
if self.clock_enabled {
// Potentially send multiple clock msgs per step if many ticks passed since last call
for _ in self.midi_beat_clock_frame .. cur_midi_beat_clock_frame {
// Send clock msg
send_live(&self.midi_output, LiveEvent::Realtime(SystemRealtime::TimingClock));
}
}
self.midi_beat_clock_frame = cur_midi_beat_clock_frame;
let timeframe = self
.timeframe_fract
.process(elapsed_since_prev_ms as f64 * self.bpm * (self.smf.ticks_per_beat as f64 / (60. * 1000.)));
let events = self.smf.events_window(self.playhead_tick, timeframe);
self.playhead_tick += timeframe;
self.callback.emit(PlayerStateChange::Playhead(self.playhead_tick));
for event in events {
// Send event to midi output
match event.event {
PlayerEvent::LiveEvent(e) => {
send_live(&self.midi_output, e);
}
PlayerEvent::Bpm(bpm) => {
self.bpm = bpm;
}
}
}
if let Some(last) = self.smf.events.last() && last.time < self.playhead_tick {
// Finished playing
}
}
self.m_prev_timestamp = Some(timestamp);
}timeframe_fract is of type TimeframeWithFract to account for fractional tick remainder, defined like this:
// Return int timeframe but add unused fractional part to next frame's timeframe
pub struct TimeframeWithFract {
fract: f64,
}
impl TimeframeWithFract {
pub fn process(&mut self, timeframe: f64) -> PlayerTickTime {
let timeframe_f = timeframe + self.fract;
let timeframe_i = timeframe_f as PlayerTickTime;
self.fract = timeframe_f - timeframe_i as f64;
timeframe_i
}
}What do you think? :)
If it's out of scope for this library, would it be possible to let user code handle the timing, so that a user can write their own web worker which sends midi messages to the JZZ player in the main app (currently the Web Midi API is not accessible for web workers, but they can be used for timing, but they have to send messages to the main app which can then send midi out.
If I wanted to do this with JZZ, would it be possible with the JZZ GUI Player? So that I can have its GUI functionality and API, but let the web worker do the timing. Like inversion of control, my app would call JZZ to advance its state, instead of JZZ using setInterval, would that be possible somehow? :)
I'm asking because this issue is very important to me, I really need midi playback continuing when working in the DAW while the browser window is not focused.
Actually there seems to be a way to run a constant FPS loop without web worker, using the Web Audio clock:
https://github.com/sebpiq/WAAClock#waaclockjs
const audioContext = new AudioContext();
const clock = new WAAClock(audioContext);
clock.start();
const event = clock.callbackAtTime(function() { console.log('step') }, 0).repeat(1./60.); // Repeat at 60 FPS
// ...
event.clear();It seems it can be used as a drop-in replacement for setInterval for any interval, not just audio related things.
Also interesting blog posts here:
https://loophole-letters.vercel.app/web-audio-scheduling (Testing different approaches.)
https://sonoport.github.io/web-audio-clock.html
https://web.dev/audio-scheduling
And this lib (mentioned here: sebpiq/WAAClock#20 (comment)) seems promising:
https://github.com/chrisguttandin/worker-timers
A replacement for setInterval() and setTimeout() which works in unfocused windows.
Exactly what's needed :)
Thank you! That's a nice feature to implement when I have more time.
Using Web Worker seems to be the most reliable solution.
Please try if the latest release JZZ v1.5.4 works fine for you.
And thanks again for the links, they were extremely helpful!
A minor bug fixed in v1.5.5.