/sonosthesia-unity-demo-midi

Primary LanguageC#MIT LicenseMIT

sonosthesia-unity-demo-midi

This Unity application demonstrates the sonosthesia MIDI packages, which are part of a wider set of packages aimed at facilitating the creation of immersive, interactive audio visual art. These modules provide an abstraction layer for MIDI messages, with a human readable API for MIDI input and output. The core message types and abstract input and output APIs are provided by

While concrete implementations are provided by

Installation

Note that to add those packages to your Unity project you will need to add the following scoped registeries to your Packages/package.json file (Keijiro only necessary for com.sonosthesia.rtmidi).

"scopedRegistries": [
    {
      "name": "Neuecc",
      "url": "https://package.openupm.com",
      "scopes": [
        "com.neuecc.unirx",
        "com.cysharp.unitask"
      ]
    },
    {
      "name": "Keijiro",
      "url": "https://registry.npmjs.com",
      "scopes": [
        "jp.keijiro"
      ]
    },
    {
      "name": "Sonosthesia",
      "url": "https://registry.npmjs.com",
      "scopes": [
        "com.sonosthesia"
      ]
    }
  ]

Scenes

UI test and monitoring tools which plug into the abstraction layer API are contained in the Assets folder of the application itself.

MIDI I/O Scenes

RtMIDIInput and RtMIDIOutput

Platform support is limited to Windows, macOS and Linux due to the dependency on jp.keijiro.rtmidi

RtMIDIInput

Listen to MIDI input port messages on the local machine using the RtMIDIInputStream component from com.sonosthesia.rtmidi.

RTMIDIOutput

Send messages to MIDI output port on the local machine using the RtMIDIOutputStream from com.sonosthesia.rtmidi.

TimelineMIDIInput and MergedTimelineMIDIInput

Supports all platforms. Generate MIDI messages from midi file tracks using the Unity timeline. Uses the TimelineMIDIOutput Implemented in com.sonosthesia.timelinemidi.

PackRawMIDIInput and PackRawMIDIOutput

Supports all platforms. Connect to a running sonosthesia-daw-connector to send and receive MIDI messages from a remote machine (including mobile and VR platforms). Follow setup and config instructions and ensure that the installed major/minor version matches the com.sonosthesia.pack package version in your unity project. You can ensure a specific major/minor version of the sonosthesia-daw-connector using @ without specifying patch version (the latest patch release will be installed)

npm install -g sonosthesia-daw-connector@1.3.0

MIDI input uses the PackRawMIDIInputStream

PackMIDIInput

MIDI output uses the PackRawMIDIOutputStream

PackMIDIOutput

Channels

Channels provide a higher level of abstraction to MDI notes and MPE notes by combining different types of messages to create UniRx data streams for each note. They are implemented in the com.sonosthesia.midi package.

MIDI Note Channels

MIDI note on, note off and polyphonic aftertouch messages can be combined to create a sonosthesia channel with each note represented as a separate stream of MIDINote with a variable Pressure field.

MPE Note Channels

MIDI note on, note off, control change (74), channel aftertouch and pitch bend messages can be combined to create a sonosthesia channel with each MPE note represented as a stream of MPENote with variable Slide, Pressure and Bend fields.

Sync and Transport

MIDI Song Position Pointer, Clock, Start, Stop and Continue messages can be used to synchronize Unity with a DAW. Bar, Beat and Sixteenth info is infered based on provided time signature.

Note the MIDI Sync functionality must be enabled in your DAW for the relevant MIDI output. In Ableton Live the song MIDI clock type must be used. If unsure of the required settings of your particular DAW you can use applications like MIDIMonitor to check that it is generating the required sync messages.

RtTransport and PackTransport

Provide example transport tracking with two different MIDI backends.

Known limitations