Timbre resources
This is a living document of resources to support digital instrument makers and music technologists work with and design timbre. Community contributions via pull requests are welcomed.
Communication Acoustics Lab, Centre for Digital Music, Queen Mary University of London
Audio content analysis
Timbre Toolbox | MATLAB toolbox for computing audio descriptors.
MIR Toolbox | Low and high level audio feature extraction. It is written in Matlab and includes preprocessing, classification, similarity measures and clustering functionality.
LibXtract | Libxtract can extract low level features in real time. It is written in C, Max-MSP, Pure Data, Super Collider and Vamp.
Essentia | An open-source C++ library for audio analysis and audio-based music information retrieval released under the Affero GPLv3 licence (also available under proprietary licence upon request) which has been developed by the Music Technology Group in Universitat Pompeu Fabra. Essentia was awarded with the Open-Source Competition of ACM Multimedia in 2013.
Sonic Visualiser | A free, open-source application for Windows, Linux, and Mac, designed to be the first program you reach for when you want to study a music recording closely. It's designed for musicologists, archivists, signal-processing researchers, and anyone else looking for a friendly way to look at what lies inside the audio file.
Spear | SPEAR is an application for audio analysis, editing and synthesis. The analysis procedure (which is based on the traditional McAulay-Quatieri technique) attempts to represent a sound with many individual sinusoidal tracks (partials), each corresponding to a single sinusoidal wave with time varying frequency and amplitude.
Audio Features Ontology | This ontology expresses some common concepts to represent some features of audio signals. It mainly relies on the Event ontology, in order to classify particular parts of the timeline backing an audio signal. It also supports dense features, such as chromagrams, onset detection function etc.
Machine learning and AI
SP-Tools | A set of machine learning tools that are optimised for low latency and real-time performance. The tools can be used with Sensory Percussion sensors, ordinary drum triggers, or any audio input.
FluCoMa | The FluCoMa software consists of objects for decomposing and describing audio, and for manipulating collections of sonic data by querying, matching, learning and transforming. The complete toolkit is available for Max, SuperCollider and Pure Data, and the decomposition / description tools are available for the command line.
nn~ | At its core, nn~ is a translation layer between Max/MSP or PureData and the libtorch c++ interface for deep learning. Alone, nn~ is like an empty shell, and requires pretrained models to operate. You can find a few RAVE models here, or a few vschaos2 models here. Using nn~ for PureData, RAVE can be used in realtime on embedded platforms, such as Bela or Raspberry Pi 4.
neutone.space | A platform where researchers can share real-time AI audio processing models for creators to experiment with transformative AI audio instruments.
Datasets
SOL | Ircam instrumental sound database coming from the Studio On Line project.
Nsynth | 305,979 musical notes, each with a unique pitch, timbre, and envelope.
Freesound | a collaborative collection of 618,244 free sounds
OpenMIC-2018 | 20000 audio clips with annotations of the presence or absence of 20 instrument classes
URMP | 44 pieces of orchestral recordings with note-level and frame-level annotations
MIS | single instrument notes with different playing techniques
Medley-solos-DB | an instrument recognition dataset, audio extracted from MedleyDB and solosDB
Miscellaneous
TOR | The Timbre and Orchestration Resource (TOR) is an open-access web platform dedicated to providing tools, materials, and resources for the analysis, creation and teaching of timbre and orchestration.
SpeaK (Max) & (Web) | SpeaK proposes a collaborative platform for organising, presenting and sharing sound lexicons combining words, definitions and sound examples.
Timbre & Formants (English) & (enlarged German version) | A collection of interactive audio examples by Christoph Reuter at the University of Vienna’s Musicological Department.
Readings and talks
This is a small selection of conference proceedings, including video recordings of talks, and books that introduce contemporary academic discourse on timbre.
Nothing but Noise: Timbre and Musical Meaning at the Edge | First book to present an interdisciplinary theory on the contribution of timbre to musical meaning and affect.
The Oxford Handbook of Timbre | First large-scale edited volume devoted to the study of timbre. Brings together philosophical, historical, cultural, technological, and analytical perspectives.
Timbre: Paradox, Materialism, Vibrational Aesthetics | This book also presents a multi-disciplinary approach to timbre, assessing the acoustic, corporeal, performative, and aesthetic dimensions of tone color in Western music practice and philosophy.
The Race of Sound: Listening, Timbre, and Vocality in African American Music | Timbre might not be as innocent as you might think. Nina Eidsheim examines singers Marian Anderson, Billie Holiday, and Jimmy Scott as well as the vocal synthesis technology Vocaloid to show how listeners carry a series of assumptions about the nature of the voice and to whom it belongs.
Timbre: Acoustics, Perception, and Cognition | First edited volume dedicated to a comprehensive and authoritative presentation of timbre perception and cognition research and the acoustic modelling of timbre.
The Relentless Pursuit of Tone: Timbre in Popular Music | First book dedicated to the subject of timbre in popular music, from San Francisco's early rave scene to the twang of country banjos and the sheen of hip-hop strings.
The Orchestral Revolution: Haydn and the Technologies of Timbre | By focusing upon Joseph Haydn's innovative strategies of orchestration and tracing their reception and influence, Emily Dolan shows that the consolidation of the modern orchestra radically altered how people listened to and thought about the expressive capacity of instruments.
From Music to Sound: The Emergence of Sound in 20th- and 21st-Century Music | Makis Solomos shows how new music, from Debussy to electronica, began a change of listening paradigm, moving from a culture centred on the note to a culture of sound.
The International Conference on Timbre | A new triennial interdisciplinary forum for exchanging novel perspectives and forging collaborations across different disciplines to help address challenges in our understanding of timbre from empirical, theoretical, and computational perspectives.
Timbre Topologies - Perceptual and Conceptual Spaces | This symposium took place at the Zurich University of the Arts in February 2023, with a focus on spatial models for timbre.
Analysis and Description of Timbre in Fixed Music | A mixture of music theorists, psychologists, and technologists came together in February 2021 at the Université Jean-Monnet Saint-Étienne in France to talk timbre.
Timbre 2018: Timbre is a Many-Splendored Thing | In July 2018 the major timbre and orchestration researchers from around the world came together in Montreal for a conference about the study and deployment of timbre in music. [proceedings] [videos of research keynote lectures]
Berlin Interdisciplinary Workshop on Timbre | In January 2017 young and established timbre researchers gathered at the Berlin Museum of Musical Instruments to assess the current state of the art in multi-disciplinary research on timbre.
Workshop on Musical Timbre | An informal, 1-day workshop at Télécom ParisTech in November 2014, aimed at bringing together researchers in MIR, music cognition, musical acoustics, audio signal processing, computer music and related fields, to discuss mutual interests, foster community, and encourage collaboration.