TimeLines: A Live Coding Modular Synth & Sequencer

A versatile, customizable, and live codable modular synth and sequencer. See it in action celebrating 15 years of TOPLAP.org!

http://www.youtube.com/watch?v=dsHnWE6_JbE

Status Update

05/06/2021

Development of TimeLines has been in hibernation for a while now, but the project is certainly not dead. The long-term roadmap includes goals like a custom language, a domain-specific editor, a purpose-designed keyboard and other hardware controllers, visual feedback and more. Currently, development efforts are focused there. More details to follow soon.

What is it?

TimeLines is a powerful, versatile, and customizable modular synth and sequencer in the form of a live coding language. This means that instead of patching cables, moving sliders, and turning dials, one writes and modifies code (which is just maths in disguise) to describe how each synth’s parameters should be changing over time.

You can think of it as the automation feature of modern DAWs on steroids, only instead of painstakingly clicking and dragging to get that shape just right, you use maths to do the job for you.

TimeLines is based on the idea that all music is a function of time. This simply means that, ultimately, music is what emerges as time flows forwards, bringing about all changes in sound that we perceive as melodies, rhythms, chord progressions etc. TimeLines builds on that concept by forming all musical structure as functions of time. Time is not dealt with behind the scenes, as is the case for most synthesizers and music software, but is rather brought to the surface and subjected to all kinds of transformations: we can stretch it, compress it, loop it, turn it into discrete steps, speed it up and slow it down, and give it all sorts of funny shapes.

Every parameter of a musical process (synth, effect, sequence etc.) is controlled through an equation that asks a simple question: “If the time right now is ‘t’ seconds (counting from the beginning of the piece or performance), what value should this parameter have right now?”. This question is then answered for every consecutive moment in time, essentially mapping out the “automation line” of that parameter. This allows one to become arbitrarily specific when making music, gaining full control over each and every aspect of their sound and composition.

The best part? The level of maths required is taught in schools, with the overwhelming majority of the equations using just addition, subtraction, multiplication, division, and modulo operations. All you have to do is learn how each of these changes the sound and behavior of your instruments.

More Technical Bits

TimeLines is a real-time environment for live coding music as functions of explicit time. It is embedded in Haskell, a purely functional programming language, and currently lives within the Emacs programmable editor. It acts as a front-end to the SuperCollider synthesis server, modulating all parameters of a synth (such as its amplitude, frequency, filter cutoff etc.) over time by sampling mathematical equations hundreds or thousands of times per second.

At the moment, TimeLines is used as the “brains” of the SuperCollider synthesis engine, a completely modular environment for building and connecting sound synthesis and editing processes. TimeLines plugs on top of those processes and hijacks their behavior, taking full control of their parameters over time. Synths can be created, freed, and patched into one another in all sorts of different ways, making TimeLines a fully modular synth and sequencer. Everything is purely functional: no state is changed, no envelopes are triggered, no sequencers are manipulated, just waves that dictate the motion of each parameter.

Currently TimeLines is being developed to also control and sequence hardware modular synths.

Installation

Disclaimer: The installation process is subject to change. In principle it should work on all major OSes (Linux, macOS, Windows), but it hasn’t been tested thoroughly yet. If you encounter problems, please email us at lnfinitemonkeys@tuta.io (mind the first letter) and we’ll fix it.

Quick (if you know what you’re doing)

To use TimeLines you will need the following installed:

Git. If you don’t have it already, you should install it first as it will be used for the others.
SuperCollider and the TimeLines Quark.
Haskell Platform (required for Stack and MSYS2 on Windows).
Emacs and the timelines-mode. (support for more editors coming soon!)
Important: You will also need the C library libsndfile installed. Installation might vary for different operating systems. See here for more information on Linux (https://stackoverflow.com/questions/2057395/package-libsndfile-dev-has-no-installation-candidate#2057481), MacOS (http://macappstore.org/libsndfile/) and Windows (http://www.mega-nerd.com/libsndfile/#Download).

In-Depth (for the rest of us)

Install Git, SuperCollider, Emacs, and the Haskell Platform (should be pretty straightforward on all major OSes).
Clone this repository to a directory on your machine (by default your home directory) using something like git clone https://github.com/lnfiniteMonkeys/TimeLines.git ~/timelines. Take note of that path.
Windows Users should use something like git clone https://github.com/lnfiniteMonkeys/TimeLines.git %HOMEPATH%\timelines instead.
Install the TimeLines Quark by opening Supercollider and evaluating the line Quarks.install("https://github.com/lnfiniteMonkeys/TimeLines-SC.git"); (you can evaluate single lines by pressing Shift + Enter). Verify that it was successfully installed by running TimeLines.start; and looking for a boot confirmation in the console.
Lastly, install the timelines-mode for Emacs by cloning it into a directory (for example Git clone https://github.com/lnfiniteMonkeys/TimeLines-Emacs.git ~/timelines-emacs) and then adding the following lines in your .emacs or .emacs.d/init.el, replacing the dummy paths for where you cloned the repos above:
```
;; Tell emacs where it can find timelines-mode
(add-to-list 'load-path "~/path/to/timelines-mode")
(require 'timelines-mode)
(setq timelines-path "~/path/to/timelines/source")
    
```
(Windows Only)

Usage

Once all of these are successfully installed, the following steps will get you up and running with a session:

Start Supercollider and start the server (see boot script). You should see a message confirming that TimeLines has been successfully booted.
Start Emacs, create and navigate to a file ending in .tl (e.g. by pressing C-x C-f in Emacs and typing the name of the file), and press C-c C-s to start a session. More keyboard shortcuts shown below.
Type some code and execute it by pressing C-ENTER. See below for examples that you can copy-paste to make sound.

Main Concepts

Signal

Signals are the building blocks of music in TimeLines, defined as functions that take time and return a value. These signals are not what comes out of the speakers, but they are used to control all the parameters of the instruments. Digital or analog, discreete or continuous, signals make the world go round.

There are, give or take, five main types of signals:

Constant: The simplest type of signal, which completely ignores the time and always returns the same value. Signals like 2, pi, and 5/4 are all constant.
Identity: The most important type of signal, time itself. Abbreviated as t, this signal will always return the value passed to it, practically acting as a clock.
Periodic: Signals that repeat their behaviour after a certain amount of time. This includes anything from simple trigonometric functions such as sine and cosine, phasors (i.e. ramps that go from 0 to 1 and repeat), or more complicated signals such as whole melodic phrases or rhythms that repeat after some time.
Pseudo-random: Any signal whose output seems random to the human ear. As opposed to random number generators, these signals won’t just yield a pseudo-random number every time they’re called, but rather have to be explicitly indexed into to get the next (or previous) values. This may sound like an unnecessarily tedious way of doing things, but it has some major advantages. For example, previously indexed values can be re-used at any time, simply by passing the same argument to the function. More on this in the examples.
Arbitrary: Lastly, the majority of signals will not fall in any of the above categories, but will be arbitrary combinations of one or more of them. Such signals may be constant for a while, then loop for some time before introducing some randomness, and finally falling constant for the rest of time. Arbitrary signals can be put together by taking a few signals of varying behaviours, cropping them so that they only have a non-zero value inside their allocated time slot, and finally summing them all together. The end result creates the illusion of a single signal, whose behaviour seamlessly (or abruptly) changes from one signal to another over time.

Timeline

A timeline is a collection of signals. They each may be of different types, have different contents and durations, and only affect the final output at certain times and not others. Ultimately, a timeline describes the life course of each parameter of every instrument in a piece of music.

Window

A Window is a frame of time. All signals are defined over continuous, infinite time. A global Window determines the time interval over which all signals are actually being evaluated and observed. Without a Window, nothing actually gets calculated, everything is hypothetical (and completely deterministic, so you can be sure that evaluating the same code over the same Window will always yield the same results).

If you are familiar with DAWs, you can think of a Window as the selected section of the piece you are currently working on, which is usually either looped, to monitor changes to it while they’re applied, or played in between changes.

If you are more familiar with programming, you can think of the Window as the viewport in a game: the code to put together a whole level already exists, but at any given moment there is only a certain window that has to be loaded and rendered, the one that the player is actually looking at. Everything else remains in the hypothetical realm, ready to be assembled when the time is right.

Synths

(WIP) A synth represents a collection of signals, each of which is assigned to a specific parameter of a sound generating and/or processing module. In other words, a synth can be a single oscillator, a filter, a delay, a reverb, a mixer, or all of the above. In fact, synths can even ignore sound altogether and instead send MIDI or OSC messages to other software or external hardware.

Patch

(WIP) A patch represents a routing connection between one or more synths, similar to buses in DAWs or actual patched connections in modular synths.

Session

(WIP) A session provides a context for all of your synths and signals. There are two main types of sessions, inspired by linear DAW timelines and traditional live coding methods respectively:

Finite Session: You specify a window, say (0, 5) or (2*barDur, 4*barDur) for some value of barDur in seconds, and all signals are only evaluated for that window.

You can think of it as selecting a section of time in a DAW: you can play it once, loop it, or change all of its parameters while its playing for instant feedback. Ideal for working on a section of a track for some time and then being able to come back and find it exactly how you left it.

Infinite Session: Signals are being evaluated in chunks of 0.5 seconds (can be changed) and the window is constantly increasing behind the scenes. Practically, this means that you can write code that does something different every (milli)second, forever. Pretty neat if you ask us.

Code Examples

{-
All examples are using a finite session with a window of 5 seconds.
You can change the time window by playing with the number in the parentheses,
or you can switch to an infinite session simply by replacing the top line of
each example to "infiniteSession $ do".
-}

-- An FM synth whose parameters stay constant over time
finiteSession (0, 5) $ do
  synth "staticTone_fm" $ do
    "amp" <>< 0.1
    "freq" <>< 200
    "modRatio" <>< 2
    "modAmt" <>< 100
    "pan" <>< 0

-- Using a sine LFO of time-varying frequency to modulate
-- the frequency and amount of modulation
finiteSession (0, 5) $ do
  synth "wobble_fm" $ do
    let lfoSpeed = goFromTo 2 8 $ t/10 -- goes linearly from 2 to 8 Hz over 10 seconds
        lfo = sin $ 2*pi*t*lfoSpeed
    "amp" <>< 0.1
    "freq" <>< 300 + 100 * lfo
    "modRatio" <>< 10
    "modAmt" <>< 500 + 300 * lfo
    "pan" <>< 0

-- Playing a looping melody while applying a tremolo LFO to the
-- amplitude and a slowed down version to the modulation amount.
-- The result is then patched into a delay synth, using the same
-- LFO to slightly modulate the delay time.
finiteSession (0, 5) $ do
  let fundFreq = 120
      -- this will loop through the semitones every 5 seconds
      melody = fundFreq * (semitones $ fromList [0, 0, 5, 7, 8, 4, 12, 12] $ wrap01 $ t/5) 
      tremoloLFO = sin01 $ 2*pi*t*6
  synth "tremoloMelody_fm" $ do
    "amp" <>< 0.1 * tremoloLFO
    "freq" <>< melody
    "modRatio" <>< 5
    "modAmt" <>< 1000 * (slow 2 $ tremoloLFO)
    "pan" <>< 0
  "tremoloMelody_fm" ><> "delaySynth_delay"
  synth "delaySynth_delay" $ do
    "amp" <>< 1
    "delayTime" <>< lerp 0.99 1.01 $ slow 4 tremoloLFO
    "decayTime" <>< 5
    "pan" <>< 0

lnfiniteMonkeys/TimeLines-hs