/AudioProgramming

Audio Programming

Audio Basics

Advice for getting into audio programming

More audio programming advise

Audio related videos (tutorials, talks, etc...)

Audio Gold Nuggets

  • The CD-quality sample rate is 44.1kHz, which means there needs to be 44100 samples per second per channel sent to the audio hardware for playback. Rather than being passed to the audio hardware a single sample at a time, the samples are passed in buffers — or blocks — containing a certain number of samples. For example, at 44.1kHz and a block size of 441 our AudioAppComponent::getNextAudioBlock() function would be called 100 times per second. (From JUCE Tutorial)

  • Just download the Ableton Link sample code, the Audiobus examples or the Apple AU sample code and you get something that works right out of the box. Added benefits: you learn exactly how stuff works because it's not some mysterious black box and you won't get weird license restrictions getting in the way."

Courses

Books

  • The Theory and Techniques of Electronic Music (Miller Puckette)
  • The Audio Programming Book (Richard Boulanger/Victor Lazzarini)
  • Programming Electronic Music in Pure Data (Johannes Kreidler)
  • Electronic Music and Sound Design V1 & V2 (Alessandro Cipriani/Maurizio Giri)
  • Musimathics: The Mathematical Foundations of Music (MIT Press) (Volume 1) (Gareth Loy)
  • Musimathics: The Mathematical Foundations of Music (MIT Press) (Volume 2) (Gareth Loy)
  • Designing Software Synthesizer Plug-Ins in C++: For RackAFX, VST3, and Audio Units (Will Pirkle)
  • Designing Audio Effect Plug-Ins in C++: With Digital Audio Signal Processing Theory (Will Pirkle)
  • Physical Audio Signal Processing: for Virtual Musical Instruments and Digital Audio Effects (Julius O. Smith III)
  • Free at: https://ccrma.stanford.edu/~jos/pasp/pasp.html
  • Spectral Audio Signal Processing (Julius O. Smith III)
  • Introduction to Digital Filters: with Audio Applications (Julius O. Smith III)
  • Mathematics of the Discrete Fourier Transform (DFT): with Audio Applications ---- Second Edition (Julius O. Smith III)
  • Audio Effects: Theory, Implementation, and Application
  • DAFX: Digital Audio Effects
  • Numerical Sound Synthesis: Finite Difference Schemes and Simulation in Musical Acoustics 1st Edition by Stefan Bilbao (Author)

Open Source CS resources

Languages

Make music with code

Frameworks and libraries

JUCE tutorials

DSP

Math refreshers

Physics of Sounds

C++ audio

iOS music app development

Good Advice for iOS Development

  • Advice from Brambos: "If you're staying on iOS, I would avoid using 3rd party APIs and simply stick to the system audio APIs. That way you can at least fix things when Apple breaks something in an iOS update without you having to wait for others to fix the frameworks for you." ** "The biggest trick is to learn how to eat an elephant: in many tiny little bites In other words: learn how to break down the project into a collection of many small problems that need to be solved one by one. It will make the learning curve more manageable and rewarding as it will give you a long chain of gratifying victories"

  • More advice from Brambos: "Making something that produces sound is probably not too hard (altough quite a bit harder than making a website with a Wordpress template).

Making a decent music app is really hard. There are a couple of things you need to master:

Programming the app (the basic framework of an app, coding the UI behavior and handling of user interaction). Ideally done in Objective-C, mixed with lots of C/C++ for the realtime audio bits. Swift is not yet a real option since most low-level frameworks are still built around (Objective-)C. Also, you need to be good at optimizing code in really clever ways if you have any hopes of doing stuff in real time with low latency.

DSP: generating and manipulating sound. Nothing about this is easy. If something seems simple, you're probably doing it wrong and soon you're in for a world of aliasing and distortion :D Lots of physics and maths involved here. Be prepared to put lots of hours into studying highly arcane aspects of physics and signal theory and brushing up on your trigonometry. Again, knowing the physics is not enough. You need to be able to build your DSP code so efficiently that it can be run in realtime without cutting corners wrt sound quality. You and Mr. Niquist are going to be very close friends.

Design: underestimated by many app developers. It can be the little things that make or break a workflow. And that's before you even get into the aesthetics, established interaction design patterns and how to deal with device fragmentation. Particularly hard on iOS because no two workflows are the same and people expect you to consider even the most obscure use cases of how your app will be used in conjunction with a chain of other apps.

The crazy intricacies of low level audio tech, i.e. CoreAudio, AudioUnits, CoreMidi, Ableton Link, etc. Many aspects of this are barely documented and need to reverse engineered by looking at C header files and labyrinthine sample code. Sometimes bugs in the OS (or third party libraries/apps) will have you pulling out your hair for days. And you have to understand low-level system stuff like threading priorities and locking to make sure your fancy GUI doesn't clash with realtime audio processing causing inexplicable glitches and clicks.

Sorry if this sounds off-putting, but there really is nothing easy about it if your ambition is to go beyond the point of an app that just plays a WAV file when you press a button on the screen B)

If you've already covered some of the above to some extent you have a pretty good starting point, however, and everything can be learned if you put your heart into it. If you're starting from scratch I would recommend beginning with something a bit simpler. :)

To counterbalance with a slightly more encouraging suggestion: I would start with a MIDI app because that would get the nasty real-time DSP out of the equation and be a more gentle introduction into the world of iOS music.

Still not trivial, but a more realistic goal for a novice project :)"

  • More advice: "A basic delay, flanger or distortion effect though? Probably a lot easier. The guts of the C/C++ code for the core DSP algorithms are out there. Sort out how to wire it up. Then add stuff. If I'm not mistaken, this is how Holderness got started! For sure, the somewhat obtuse notion of 'a good flanger' will likely not come from pulling a repo on github. But it can get you started if you're interested. A little more complicated but not in the Phasemaker realm: a basic VA subtractive synth. One OSC, a LPF and an envelope where the pitch of the osc is mapped to a keyboard. There are tutorials out there for this. And these days you can cook one up with Javascript. Will not be of much practical use on iOS but the core concepts are the same and you can learn on any OS without needing to deal with compilers."

Programming Audio Units

iOS Audio apps

AVAudioEngine

iOS frameworks

AudioKit

Tutorials

MAX related

GEN

Arduino Audio

Web Audio

Good Blogs

Podcasts

Great Projects

Arduino base

littleBits

Employers - besides the usual suspects and major audio/music complanies

Resources

Misc

Sound Design

Ideas/Interests

  1. algorithmic/generative music (in addition: processes that ‘tracks’ parameters)
  2. build interactive synthesis objects that allows OS to control sound parameters dynamically
  3. Start several distinct music processes that respond to actions upon particular game objects/user interactions

Focus areas/Goals for music technology (taken from Calarts website)

The Music Technology program aims to built strong musical skills while promoting the mastery of a variety of music-related technologies in the context of pursuing creative work. Specifically, Music Technology students will have opportunities to:

  • become trained musicians able to work in a variety of musical ensembles and global music productions, with appropriate competencies in music theory and musicianship skills, while emphasizing the specialties of 21st Century music technology;
  • develop professional-level skills allowing them to produce concerts, stage-manage, understand sound mixing and diffusion, and run stage monitoring and main audience sound;
  • become skilled in and know how to run music studios at a high level for album and media production, including microphone techniques, software editors, audio effects, mixing, mastering, and the use of the Internet for audio production;
  • develop thorough historical knowledge of electro-acoustic music as well as knowledge of the theory and operation of algorithms for traditional synthesis and audio effects production;
  • become software engineers able to write computer code for websites for the Internet and understand advanced object-oriented computer languages for artistic expression;
  • learn to design and build basic electronic circuits and make human computer interface designs for artistic practices;
  • acquire knowledge of basic digital signal processing and how it relates to audio, including time and frequency domain processing and how these may be manipulated for artistic practices;
  • develop high-level practical and professional skills, including the ability to synthesize diverse studies, project planning, execution, time management, and documentation of both technical and musical work at a level suitable for publication in a professional journal.

Music technology MFA degree requirement

Program Requirements

  • MTEC600 Topics in Music Technology (2 classes)
  • MFOR506 Creative Technology Forum (4 classes)
  • MTEC613 & MTEC614 Introduction to Programming I & II (2 classes)
  • MTEC621 C++ for Electronic Music I (1 class)
  • MTEC630 & MTEC631 Interface Design I & II (2 classes)
  • MTEC660 Audio Signal Processing (1 class)
  • MTEC685 Teaching in Technology (2 classes)
  • MHST525 Survey of Sound Art (1 class)
  • MTEC690 Music Technology MFA Project (variable units; must enroll each semester in residence)
2 Music Technology electives, chosen from (2 classes):
  • MTEC580 Advanced Web Application Development
  • MTEC680 Music Information Retrieval & Machine Learning for Art
  • MTEC616 Visual Programming
  • MTEC550 Robotic Design for Music
  • MTEC612 Mobile Music Computing
  • MTEC622 C++ for Electronic Music II
  • MTEC635 Advanced Circuit Design
  • MTEC640 Programming for Raspberry Pi
  • MTEC650 Sound Synthesis
  • MTEC655 Advanced Synthesis
2 Digital Performance Ensemble electives, chosen from (2 classes):
  • IIMC550 SoundGameSpace
  • MBLE625 Creative Electronic Ensemble
  • MCMP615 Choreographers & Composers
  • MTEC540 Machine Orchestra
  • MTEC618 Data Driven Art & Visuals
  • MTEC619 Experience Design & Technology
  • MTEC520 Grids, Beats, and Groups
  • MTEC530 21st-Century Raga & Tala
  • MTEC541 Composition for Robots