Twitch Plays MIDI

A simple Processing/java template for collective "music" performances over Twitch. It creates a chatbot that connects to your Twitch channel and lets users interact with your computer through text commands like:

delay 90 2 - changes the controller labeled "delay" to 90% over the course of 2 seconds dropbass - sends a C2 note

TwitchPlaysMIDI was created for TwitchPlaysBees. It controlled two VST filters in Ableton Live. The VST filters windows were on a second display with a green background and overlayed on the stream with a chroma key effect.

Requires Processing a free Java-based environment for creative coding.
Uses PircBot an IRC library for java (included in code/)
Uses The MidiBus a Processing library (not included)
Uses Ani a Processing library that manages eased transitions between values

Setup

Traditionally Twitch-plays games run on dedicated machines that are streaming the output of the game through OBS or other applications. This program listens to Twitch chat messages and interprets them as MIDI input on your computer. A Twitch chat mostly functions like an IRC channel, so your bot can both receive or send text messages. You need some familiarity with MIDI configuration to set this up.

Download Processing (from the link above) and the code of TwitchToMIDI (from GitHub/this page)
Open the project in Processing making sure that all the files are in a folder named TwitchPlaysMidi (not TwitchPlaysMidi-master).
Add your Twitch credentials (channel name, Oauth) to Bot.pde. If "irc.twitch.tv" in Bot.pde doesn't connect try "irc.chat.twitch.tv" (located further down in the code)
Install The MidiBus and Ani libraries from Processing with Add Library...
Set up your virtual MIDI bus/port. On Window you will need a Virtual Midi Cable program like loopMIDI (launch, add port with the +). On Mac you need to follow these instructions
Run the sketch check the list of MIDI device and replace "loopMIDI Port" in TwitchToMidi.pde with the device you want to use. Also check if the bot puts the "succesfully connected" message in your stream chat to make sure everything works till now.
In TwitchPlaysMidi.pde add your own chat commands and related MIDI controllers in the arraylist "controllers"

Trigger

Code: triggers.add(new Trigger("[CommandName]", "[note]"));

[CommandName] - the exact phrase that when put in the chat should trigger the MIDI (you have to add ! infront of it if you want it to look like a chatbot command)
[note] - the MIDI note you want to be send

Example: triggers.add(new Trigger("dropBass", "C4"));
If dropBass is send in chat the MIDI note C4 will be send to your DAW and trigger that note on your current instrument

Slider

Code: controllers.add(new Slider("[CommandName]", [channel], [minValue], [maxValue]));

[CommandName] - the exact phrase that when put in the chat should trigger the MIDI (you have to add ! infront of it if you want it to look like a chatbot command)
[channel] - the coresponding channel for the setup as a number from 0 upwords (see below)
[minValue] - the smalles value this slider should be allowed to be set to between 0-100 (it wont go lower than this value or 0)
[maxValue] - the largest value this slider should be allowed to be set to between 0-100 (it wont go highter than this value or 100)

If no [minValue] & [maxValue] is set it will be 0 & 100 respectively

Example: controllers.add(new Slider("!delay", 0, 0, 25));
If !delay 20 2 is send in chat the according Slider will be set to a value 20 over the cause of 2 seconds
If !delay 30 5 is send in chat the according Slider will be set to a value 25 over the cause of 5 seconds (because of the set maximum value)

Set up your music making program (DAW) and map the controls you want to expose to remote MIDI. This configuration depends on the application. Examples: Ableton Live, FL Studio, Cubase... the setup with Cubase has not been tested

Setting up:

Trigger: you can just type the command in chat and it will send the MIDI to your DAW so it can be used for the setup
Slider: at the top of TwitchToMIDI.pde there is a CONFIGURE variable that, if set true, will send a MIDI signal everytime you click on the window of the running sketch (black window that pops up when you click start). By default this will start with channel 0 and then count up for every click, so you have to keep in mind which effect you assignt to which channel. If you make a mistake, just re-run the sketch and do the steps again.

Test the Twitch to MIDI communication first and then the MIDI to DAW communication. You don't have to be live on Twitch to test the chat functionalities, just open your offline channel and type in the chat.
Start streaming
???
Profit!

Notes

This program doesn't create any music, it only produces MIDI inputs emulating a keyboard or a controller. You will need a MIDI-enabled digital audio workstation like Ableton Live, Garage Band, or FL Studio to turn notes into sound.
On Twitch there is a lag of a couple of seconds between video and chat, don't expect users to be in sync or to make meaningful compositions. TwitchPlaysMIDI is probably more appropriate for hybrid performances in which users contribute to a live set, slow ambient pieces, or generative music.
Twitch hides the channel description at the bottom of the page so users don't generally see the instructions. Consider setting up a stream overlay with some information on how to interact. You can use Processing's graphical capabilities for that.
If you don't receive any MIDI in your DAW, check if your Virtual Midi Cable program has stopped working. loopMIDI for example has a function that stops any given MIDI port, if too much data is sent through it in a short time, which could mean you created a loop within TwitchToMIDI. If you don't need TwitchToMIDI to receive any MIDI, please leave the input section in the config empty to prevent such loops. If you should need TwitchToMIDI to receive MIDI, please use 2 different virtual MIDI ports (one for input and one for output)

GregHattJr/TwitchToMIDI

Twitch Plays MIDI

Setup

Notes