Functional Video Sequencer
Like a mod-tracker, but:
- For timing and blending 2D video instead of sound, and
- Using generative, pixel-shading functions for the 'samples' instead of pre-recorded videos
An Effect is a function like a pixel shader: it gets fed an x coordinate, a time, and (optionally) custom arguments, and it produces RGBA values for an entire light strand. You can have a maximum of 128 different Effects.
An Event is an instance of an Effect. It specifies a time to start, a the strands it affects, and optional custom arguments. Events reference Effects by numeric identifier.
A Presentation is the entire ‘movie'. It specifies a BPM it is associated with, and an ideal reference song to play with it. A Presentation has a Timeline that groups together all Events to display. Presentations can be played by the runtime along with a different song at a different BPM, speeding them up or slowing them down.
Note: Effects are code in the host runtime, and not stored within a Presentation.
{ "bpm" : 100.32, "length" : 237483, "media" : "CHVRCHES-LeaveATrace.m4a", "timeline" : [ [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }], [{ "effect":2, "speed":2.5 }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":2, "speed":2.5 }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }], [{ "effect":2, "speed":2.5 }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[255,0,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":0, "start":0 }, { "effect":1, "start":598, "args":[0,255,0] }], [{ "effect":2, "speed":2.5 }] ] }
All multi-byte numbers are stored in little-endian format.
| bytes | field |
|---|---|
| 4 | Presentation BPM (float) |
| 4 | Presentation duration (uint32) |
| 1 | Characters in Media URI (uint8) |
| ~ | Media URI (UTF-8) |
| ~ | Timeline Index |
| ~ | Timeline |
| bytes | field |
|---|---|
| 1 | Strand Count (uint8) |
| 4 | Offset to Strand 1 Timeline (uint32) |
| 2 | Byte count of Strand 1 Timeline (uint16) |
| 4 | Offset to Strand 2 Timeline (uint32) |
| 2 | Byte count of Strand 2 Timeline (uint16) |
| 4 | Offset to Strand … Timeline (uint32) |
| 2 | Byte count of Strand … Timeline (uint16) |
| ~ | (repeat for all strands) |
| bytes | field |
|---|---|
| 4 | Strand 1 Event Count (uint32) |
| ~ | Event 1 |
| ~ | Event 2 |
| ~ | Event … |
| 4 | Strand 2 Event Count (uint32) |
| ~ | Event 1 |
| ~ | Event 2 |
| ~ | Event … |
| 4 | Strand … Event Count (uint32) |
| ~ | Event 1 |
| ~ | Event 2 |
| ~ | Event … |
| bytes | field |
|---|---|
| 1 | Effect# (uint8; 128-255 only) |
| 4 | Start (uint32 ms) |
| 4 | Speed (float) |
| 1 | Arg Count (uint8) |
| 1 | Arg 1 (uint8) |
| 1 | Arg 2 (uint8) |
| 1 | Arg … (uint8) |