HarmonAIze is a MIDI sequencer application built for the Winter 2018 senior course Accessible Software Design (EECS 498) at the University of Michigan. Features of the application include:
- MIDI recording via on-screen keyboard or hardware MIDI input
- MIDI playback via SFZ instruments
- Seconds/beats/time signature mappings, allowing music-based (i.e. BPM) manipulations, such as tempo adjustment
- Arrangement view for viewing the contents of the current project
- Save/load, undo/redo, font size adjustment.
In addition to these, HarmonAIze's "headliner" feature (and namesake) is an accompaniment generator which can intelligently generate accompaniments based on the user-created melody in the first track.
The application portion of HarmonAIze is built on the JUCE framework, a C++ framework suited for building interactive multimedia applications for macOS, Windows, Linux, iOS, and Android. HarmonAIze is tested to work on macOS High Sierra 10.12.4 compiled with Xcode 9.3. The C++ style strictly follows the JUCE style guide with US spelling conventions.
To work on the application, use JUCE's Projucer IDE project generator to open and edit /app/harmonaize.jucer. Information about how to use the Projucer can be found on the JUCE website.
The application is structured with a strong model-view-controller paradigm; every object instance has its data backed by an abstract node in a ValueTree object. This greatly simplifies data management, particularly when it comes to undo/redo, save/reload, inter-object communication, event handling, and app-wide debugging situations. The implementation is based loosely on concepts delivered by developer David Rowland during his technical talk Using JUCE value trees and modern C++ to build large scale applications at the 2017 Audio Developer Conference in London.
In addition to the UI/main thread's strong MVC representation, there is a runtime-only representation reflected to the real-time audio thread whenever the main thread's model changes. This is thread-safe for reading by the audio thread, and can be seen in objects such as MidiMessage (real-time) vs. MidiMessageModel (model-backed), and MidiMessageSequence (real-time) vs. MidiMessageSequenceModel (model-backed). These real-time objects are recreated in the main thread any time their model-backed counterparts are changed, and swapped out via a lock (as quickly as possible) as soon as they're ready (example).
HarmonAIze's accompaniment generation uses macOS's embedded Python 2.7 along with the mido and mma modules. Communication between the two languages is accomplished with pybind11. Glue/initialization code can be found in harmonaize/harmonaize/midiinterchange/hmnz_MidiInterchange.h.
Due to time and resource constraints, HarmonAIze is still largely incomplete and unsuitable for general distribution. The following is a list of features we'd like to have completed but didn't have time for:
- Metronome count in
- Arrangement view editing
- MIDI note editor
- Snapping/quantization
- Default instrument variation
- Asynchronous accompaniment generation (i.e. as-is now, but without blocking the UI thread)
- Audio file export
- Port to additional platforms
Good luck. All team members are still available via email if guidance is required, but please be specific when contacting us.
Audio programming is a type of soft (some, like myself, would argue hard) real-time programming which is incredibly challenging, requiring deep knowledge of systems programming including concepts of multithreading, lock-free coding, strategic memory management and object lifetime. It is not for the faint of heart or those with severe senioritis. If you are interested in reading about concepts reflected in the design of HarmonAIze, here are a few resources reflected in its code, primarily in /app/Source/hmnz_PlaybackEngine.cpp:
Real-time audio programming 101: time waits for nothing by Ross Bencia, AudioMulch
C++ in the Audio Industry by Timur Doumler, ROLI
C++ in the Audio Industry, Episode II: Floating Atomics by Timur Doumler, ROLI
The JUCE forum: many huge names in the audio software industry hang out here and are happy to exchange help and advice for non-trivial questions.
"Excellent in all regards. Really an excellent presentation and demo. As well, the project is technically sophisticated and graphically stunning."
- Dr. David C., Course Professor
"Your application is virtually flawless with a massive array of features, truly complex software and a really attractive UI. One of the best projects that this class has seen!"
- Adithya R., Teaching Assistant
"Really successful application in terms of having a wide set of features packaged together in an attractive and complete manner. The accompaniment creation is really quite impressive, and the ability to modify instruments allows for additional creativity. Tools such as the metronome help provide additional practicality to the application. Working with Brad to confirm accessability and comfort for him across the board would lead to a tremendously useful offering for Brad and overall."
- Kayla F., Teaching Assistant