Problem with pause/resume of MIDI file play.
urbantigerau opened this issue · 7 comments
IMPORTANT
Before submitting this issue
[ ] Have you tried using the latest version of the library? Yes
[ ] Have you checked this has not already been submitted and/or resolved? Not that I can find.
[ ] If you are requesting help a better choice may be the Arduino forum This is a suspected bug or a proposal for an improvement, the question is not of a general Arduino nature.
Some background:
I am in the process of building a stand-alone low cost MIDI file player. It is made up of two parts.
The MIDI file player, which reads an SD card and is controlled currently by the CLI, and produces a MIDI serial stream is the MD_MIDIFile software which feeds a separate synth part to produce the notes as sound. Currently the hardware that MD_MIDIFile runs on is an Arduino Mega using an AT1280 processor. MIDI output is on hardware serial port 1 (software serial is not used) and is set to 38400 baud (that's what the synth part wants and yes I know it isn't standard.)
The synth side is the Arduino Midi Synth v0.1
to be found here https://github.com/DLehenbauer/arduino-midi-sound-module). It running on a Pro-mini which has been re flashed to look like a Uno. The Serial port is set to communicate at 38400 baud.
This configuration is working decently. I can read the files on the SD card and when a file is played from start to finish it sounds as good as the midi-sound-module can do. It sounds the same as when I play the same file on my PC directly into the synth. I use either Audacity or a freeware DAW MuseScore 3.
I should note that my musical abilities are limited. However my usage of these pieces of software is pretty basic. It is little more than mapping outputs and play/pause. This all works just fine.
When using the CLI variant of the MD_MIDIFile software there appears to be a problem with the way the pause/resume of MIDI file play operates. Maybe I have misunderstood the way it is supposed to operate but I think the play/resume function does odd things, at least not what I expected.
I was expecting that when the play was paused the synth would silence and further play would not continue. When played from my PC directly and pause is chosen it does silence. I have been talking to the author of the synth side and I am assured that the "All notes off" channel command is implemented.
When using MD_MIDIFile CLI if I send c p1 (which I interpret as Command Pause On) from the Arduino console playing pauses by the synth continuing to play the last notes it was sent (or as near as I can tell) and it continues to play them, seemingly without limit. On sending a c p0 (Command Pause Off) playing resumes but it is somehow not right. I apologize for not being able to give a decent description of what it sounds like. But it seems to have lost synchronization across instruments or perhaps channels. Notes start and finish at the wrong times. At the end of the melody it is very noticeable. The tune just finishes wrong. Again I apologize for a poor description.
I thought I might be able to do a bit of debugging and here is what I found, for what it is worth. Care is needed here as debug outputs to the console potentially interfere with the timing of the MIDI stream.
So I set the console serial speed to 115200 and changed the code in midiCallback to look like this:
void midiCallback(midi_event *pev)
// Called by the MIDIFile library when a file event needs to be processed
// thru the midi communications interface.
// This callback is set up in the setup() function.
{
// Send the midi data
if ((pev->data[0] >= 0x80) && (pev->data[0] <= 0xe0))
{
MIDI.write(pev->data[0] | pev->channel);
Serial.println();
Serial.print(pev->data[0] | pev->channel, HEX); Serial.print(",");
MIDI.write(&pev->data[1], pev->size - 1);
Serial.print(pev->data[1] , HEX); Serial.print(",");
Serial.print(pev->data[2] , HEX);
}
else
{
MIDI.write(pev->data, pev->size);
Serial.println(); Serial.print("else ");
Serial.print(pev->size);
}
// Print the data if enabled - note that this is disabled
if (printMidiStream)
{
CONSOLE.print(F("\nT"));
CONSOLE.print(pev->track);
CONSOLE.print(F(": Ch "));
CONSOLE.print(pev->channel + 1);
CONSOLE.print(F(" Data"));
for (uint8_t i = 0; i < pev->size; i++)
{
CONSOLE.print(F(" "));
if (pev->data[i] <= 0x0f)
CONSOLE.print(F("0"));
CONSOLE.print(pev->data[i] , HEX);
}
}
}
I am happy to be corrected but what I think this does is send the same items to the console (but in HEX) as to the MIDI stream. Perhaps I could have used the internal debug capability but it just did not quite seem to do what I wanted. If the file is played end to end it works just fine. I also ran a version where I stopped the file at a point and restarted it. I copied the output from the Arduino Console into text files which have csv format, imported them into excel so the results can be compared side by side easily. If you look at the spreadsheet I have used conditional formatting to highlight differences.
The file played was the Elizabethan Serenade (apologies to Ronald Binge) and the side by side comparison can be found in full the file pauses.xlsx. Both xlsx and .mid files are included in the attached zip. The file is way too big to show here but I can show significant segments:
The data is arranged as two sets of columns. The left side shows the condition which will eventually be paused and the right columns show the output for the play which ran interrupted from beginning to end. I only show a few lines but as is clearly seen the output streams to the synth are identical.
The start looks like this:
Pause No Pause
Read File: ELIZ.MID Read File: ELIZ.MID
Set file : ELIZ.MID Set file : ELIZ.MID
OK OK
B0 7 7F B0 7 7F
B0 A 3C B0 A 3C
B0 5B 33 B0 5B 33
C0 49 33 C0 49 33
B1 7 7F B1 7 7F
B1 A 32 B1 A 32
B1 5B 36 B1 5B 36
C1 49 36 C1 49 36
B7 7 7F B7 7 7F
B7 A 28 B7 A 28
B7 5B 33 B7 5B 33
C7 49 33 C7 49 33
Now to the pause/resume example, many rows down the spreadsheet, row 623 to be exact.
Again only a few rows are shown.
8D 40 40 8D 40 40
8D 3C 40 8D 3C 40
8E 30 40 8E 30 40
OK This is the OK acknowledgement Pause on
OK This is the OK acknowledgementPause off
9C 45 2D 90 4D 3C These rows in the non-pause side have been moved down to correspond with the paused side
9D 3C 2D 91 48 3C
9D 41 2D 97 41 3C
9E 35 2D 93 45 3C
8C 45 40 9C 45 2D
8D 41 40 9D 3C 2D
8D 3C 40 9D 41 2D
8E 35 40 9E 35 2D
90 4D 3C 8F 24 40
91 48 3C 9F 29 2D
97 41 3C 86 18 40
93 45 3C 96 1D 2D
9C 45 2D 80 4D 40
9D 3C 2D 81 48 40
9D 41 2D 87 41 40
9E 35 2D 83 45 40
8F 24 40 8C 45 40
86 18 40 8D 41 40
80 4D 40 8D 3C 40
81 48 40 8E 35 40
87 41 40 9C 45 2D
83 45 40 9D 3C 2D
8C 45 40 9D 41 2D
8D 41 40 9E 35 2D
8D 3C 40 8C 45 40
8E 35 40 8D 41 40
I notice two things.
- The MD_MIDIFile player does not send a sequence to silence the synth
- Once resumed there seems to be no correspondence anymore between the non-paused columns and the paused. I would have expected that the MIDI streams would have been identical except for the extra silencing instructions, just shifted in time.
Finally here are the ending bytes:
8E 29 40 80 4D 40
80 4D 40 81 48 40
81 48 40 87 41 40
87 41 40 83 45 40
83 45 40 8B 41 40
8F 1D 40 8C 39 40
86 11 40 8D 30 40
CB 2D 40 8E 29 40
9B 41 1E 8F 1D 40
8B 41 40 86 11 40
This seems to explain why the melody sounds wrong and ends oddly.
Can you please advise if this is expected behavior and if the pause function is usable? An alternative I have tried and which works is that if the play is paused then a resume restarts the play from the beginning of the file. It isn't what most people understand by pause/resume however.
Your Environment
Library Version: V2.3.2
Arduino IDE version: 1.8.7
Hardware model/type: Arduino Mega with 1280 processor
OS and Version: Windows 10 Home 64Bit Version 1909 Build 18363.720
Steps to Reproduce
See above
Expected Behaviour
Synthesizer playing the notes being sent should go silent on pause and resume playing the melody correctly
Actual Behaviour
Synth continues to play static notes and melody plays incorrectly when resumed.
Code Demonstrating the Issue
-
The synthesizer does not go silent on pause() by design. If you want a silence add it into the application code, as is done in the other examples. CLI is meant for testing individual functions and so the silence is a separate CLI command in that example.
-
As for the strange sequencing, I can replicate that some of the time. It seems like one of the tracks (the one with the sustained sound perhaps?) is not resynching properly. This only seems to occur when there is a sustained sound during the pause. At all other times it works fine as far as I can tell. Still investigating.
I think pause() inconsistency is fixed. Released as version 2.4
To be clear, the pause() is to pause parsing the file. The application code controls the sound generation, which could be a hardware device or s simple output to serial. In my philosophy it is always better for the library to not assume things (ie, how to silence devices) and let the parts that do the actual control (ie, the application) implement what is appropriate.
In the CLI the midiSilence() function sends commands to make all the channels quiet.
Yes that is much clearer. I can go from here I think when I get a spare split second to try the new version.
Thanks again.
Good suggestion. I may also include that the application may need to silence sustained sounds.