jazz-soft/JZZ

Prevent hanging notes when switching ports in the middle of playback

Boscop opened this issue · 2 comments

I noticed whenever I switch the JZZ Player between midi and audio output, I get hanging notes.
Using this code, calling toggle_output_audio_midi() to switch ports:

import JZZ from "jzz";
import jzz_midi_smf from "jzz-midi-smf";
import jzz_gui_player from "jzz-gui-player";
import jzz_synth_tiny from "jzz-synth-tiny";

jzz_midi_smf(JZZ);
jzz_gui_player(JZZ);
jzz_synth_tiny(JZZ);

export class JzzPlayer {
	constructor() {
		JZZ.synth.Tiny.register("Web Audio");
		this._player = new JZZ.gui.Player({});
		this._preferred_ports = [
			JZZ().openMidiOut("Web Audio"), 
			JZZ().openMidiOut("loopMIDI Port"),
		];
		this._usingMidiPortInsteadOfAudio = false; // Use Web Audio initially
		this.connect_to_preferred_port(false);
	}
	load(smf_data) {
		this._player.load(new JZZ.MIDI.SMF(smf_data));
	}
	play() {
		this._player.play();
	}
	connect_to_preferred_port(disconnect_other) {
		if(disconnect_other) {
			this._player.disconnect(this._preferred_ports[+!this._usingMidiPortInsteadOfAudio]);
		}
		this._player.connect(this._preferred_ports[+this._usingMidiPortInsteadOfAudio]);
	}
	toggle_output_audio_midi() {
		this._usingMidiPortInsteadOfAudio ^= true;
		this.connect_to_preferred_port(true);
		return this._usingMidiPortInsteadOfAudio;
	}
}

How to prevent hanging notes?

Previously I wrote a midi player myself, and what I did there was:

  • Track how the output port's state changes based on sent midi events. Basically applying outgoing midi events to change the state represented by a data structure like this:
pub struct ChannelState {
	ccs: [u8; 128],
	program: u8,
	pitch_bend: u16,
	notes_on: [u8; 128], // 0 == off
	// TODO: keep track of ChannelAftertouch, PolyphonicAftertouch, RPN settings
}
pub struct PortState {
	channels: [ChannelState; 16],
}

And whenever playback is paused/resumed or the player jumps (e.g. because of looping), the "diff" between the current and desired port state is computed in terms of midi events that would need to be sent to set the output port equal to the desired port state, and the diff is then sent.
When switching ports, in terms of midi diff it's equivalent to pausing (sending the diff(playing_state, default_port_state)), then switching output ports, then resuming (sending the diff(default_port_state, playing_state)).
When jumping, I was sending the diff(playing_state, state_at_jump_target_tick), where state_at_jump_target_tick is computed by "integrating" (applying to a default_port_state) all midi msgs since the beginning of the SMF untiljump_target_tick. (For loops, the state_at_jump_target_tick` gets repeatedly used so in theory it could even be cached but for small midi files it's cheap to recompute on every loop jump.)

I'm curious if this kind of port state management would be possible with JZZ's existing callbacks/hooks.
I can register my own port state tracker (like the "Console Logger" example) but when playing a SMF file, how can I send these diffs as if from the JZZ player to the output port without them going through my custom output AND without messing up JZZ Player's internals?

Ideally, the midi signal flow would be: JZZInternalPlayer -> PortStateTracker -> MidiOut
But currently it's JZZInternalPlayer -> MidiOut and if I register my own midi output, it would additionally be JZZInternalPlayer -> MyMidiOut. So if I have my PortStateTracker in MyMidiOut and then send messages "through" the player, MyMidiOut will receive them, too, which messes up the tracker because it's supposed to track the playing state.
(Also, how to actually send messages "through the player when it's playing a SMF file?)

If there is no good way to add this in user code using the API, it would make sense to have a built-in way of preventing hanging notes and hanging/lingering CCs, Program & pitchbend (!).

The easiest solution would be to just reset the output port like this before disconnecting, although this sends more messages than necessary which takes more time (depends on the midi port/device):

// Reset notes, CCs, program, pitchbend
fn reset_port(midi_output: &MidiOutput) {
	use midly::{num::*, MidiMessage, PitchBend};

	const CC_ALL_SOUND_OFF: u8 = 120;
	const CC_RESET_ALL_CCS: u8 = 121;

	for ch in 0 .. 16 {
		let channel = u4::new(ch);
		send_live(midi_output, LiveEvent::Midi {
			channel,
			message: MidiMessage::ProgramChange { program: u7::new(0) },
		});
		send_live(midi_output, LiveEvent::Midi {
			channel,
			message: MidiMessage::PitchBend { bend: PitchBend(u14::new(0x2000)) },
		});
		send_live(midi_output, LiveEvent::Midi {
			channel,
			message: MidiMessage::Controller { controller: u7::new(CC_ALL_SOUND_OFF), value: u7::new(0) },
		});
		send_live(midi_output, LiveEvent::Midi {
			channel,
			message: MidiMessage::Controller { controller: u7::new(CC_RESET_ALL_CCS), value: u7::new(0) },
		});
	}
}

The port state tracking would have additional advantages because when connecting to a port while notes are on, they would also be sent, not just new NoteOn msgs. (I did that in my midi player, except for drum notes. This is also useful for having a non-full-song loop.)

It would also be useful to have an optional port-reset button on the player (that does something like reset_port() above).

I noticed I also sometimes get hanging notes when clicking the stop button on the GUI player.

You can use the onSelect() handle to send the allSoundOff message to the previous port