Visualizing Music with p5.js

This session is for anyone who would like to explore music, visuals and creative coding for the web. We'll demonstrate types of data we can get from digital signal processing using interactive sketches in p5.js and the p5.sound library that builds upon the Web Audio API.

Part 1: Signal Processing

amplitude
frequency
peak detection
pitch detection / autocorrelation

How can we map this data onto meaningful visuals that enhance our experience of music?

Part 2: Musical Timing

sync music to timestamped lyrics
pre-analyze file with the Echo Nest API, and visualize the result

Demos:

Amplitude | Source Code

Amplitude Logarithmic Map

Amplitude over time

Threshold

Simple Beat Detection

Draw Full Waveform w/ Playhead

Time Domain (Oscilloscope) | Source Code

FFT Spectrum Drag, Drop 'n Analyze | Source Code

FFT Spectrograph

FFT Scale by Neighbors

FFT Scale by 1/3 Octave

FFT Unknown Pleasures

FFT Particle System | Source Code

FFT Peak Detect | Source Code

Autocorrelate Time Domain | Source Code

Autocorrelation Circle | Source Code

Pitch Track | Source Code

Display Lyrics | Source Code

Pre-rendered Analysis (via Echo Nest - Beat + Pitch) | Source Code

Pre-rendered Analysis (via Echo Nest) - Pitch Segment 1 | Source Code

Libraries

Participants may use whatever tools they wish, but the demos in this repo use the following libraries:

p5.js is a JavaScript library that starts with the original goal of Processing, to make coding accessible for artists, designers, educators, and beginners, and reinterprets this for today’s web.

p5js.org
- /learn
- /reference
github

p5.sound.js is an addon library that brings the Processing approach to the Web Audio API.

p5.dom.js is an addon library that helps us manipulate the DOM.

Setup

Download this github repo, and build off of the empty example sketch in the template folder. It links to the libraries.
Running p5.js on a local server
You'll need a text editor. Some options:
- SublimeText
- p5 IDE

Getting Started with p5

Your First Sketch

A few more complex p5 examples:

p5.sound Classes That We'll Use For Music Visualizations:

p5.AudioIn - microphone! documentation | source code

p5.SoundFile - load and play .mp3 / .ogg files. documentation | source code

loadsound() creates a SoundFile using a Web Audio API buffer. Use during preload(), or with a callback, or with drag and drop.

p5.PeakDetect - detect beats and/or onsets within a frequency range documentation }| source code

p5.Amplitude - Analyze volume. documentation | source code

.getLevel() returns a Root Mean Square (RMS) amplitude reading, between 0.0 and 1.0, usually peaking at 0.5
.smooth()

p5.FFT - Analyze amplitude over time or frequency. documentation | source code

.analyze() returns amplitude readings from 0-255 in the frequency domain.
.waveform() returns amplitude readings from -1 to 1 in the time domain. demo | source

Music included in the demos/repo:

Yacht - Summer Song (Instrumental) - See Mystery Lights Instrumentals Creative Commons BY-NC-SA
Broke For Free - As Colorful As Ever - Layers - Creative Commons BY-NC
Alaclair Ensemble - Twit Journalist - This Is America - Creative Commons BY-SA
Peter Johnston - La ere gymnopedie (Erik Satie) - Best of Breitband Vol1
Inara George - Q - Sargent Singles Vol 1 Creative Commons BY-NC-SA
For more Creative Commons resources, check out the Free Music Archive's Guide to Online Audio Resources

Dig Deeper into Web Audio Music Visualization...

Pitch Detection - Web Audio Demo
Another Approach to Beat Detection Using Web Audio API
Making Audio Reactive Visuals w/ Web Audio API
Marius Watz' Sound As Data workshop with Processing // blog post
Echo Nest Remix API can get you beats, tatums, regular API has more data about music/artists/songs.
p5.gibber Rapid music sequencing and synthesis. Also its own live coding environment.
Tone.js is a JS library for composing interactive music.
dancer.js is a JS library for audio visualizations.

robstenzinger/p5-music-viz