Audio programming in game development. Everything at and above the sound buffer layer

[bigrando420]

I'm just starting out with audio programming from scratch in game development.
The layer I'm on is writing bytes to a sound buffer. I'd ideally like some clues in regards to:

• mixing
• structuring things nicely (lol ??)
• panning the audio spatially
• applying fx like pitch shifting, reverb (and what the API for this would look like)
• wtf actually is volume / dbs, how do those translate into writing -1.0 to 1.0 in the buffer?

So far I've got sokol_audio for writing into the sound buffer. My current knowledge ends at writing a sine-wave into the buffer to make a sound at a certain frequency.

TIME 2 GET LEARNING BAYBEEEEEEEE

[RealNeGate]

Someone correct me if i'm being silly n' goofy, it's been a minute:

I personally like miniaudio though it's huge so imma take some excerpts of the docs and code as reference:
https://github.com/mackron/miniaudio

Volume

wtf actually is volume / dbs, how do those translate into writing -1.0 to 1.0 in the buffer?

if you're dealing with db volume, the idea is that db (decibels) are logarithmic, they're gonna scale funny (iirc 10x volume is around 10+ decibels) and thus you wanna convert them into a nicer volume you can easily multiply against the normie -1.0 to 1.0 values.

on line 43037, we get this conversion between decibels and the nicer linear factors:

MA_API float ma_volume_linear_to_db(float factor)
{
    return 20*ma_log10f(factor);
}

MA_API float ma_volume_db_to_linear(float gain)
{
    return ma_powf(10, gain/20.0f);
}

Mixing

Mixing itself isn't fancy, it's basically just adding a bunch of waves together and sometimes multiplying them by different factors. sin(t) is one wave, sin(t) + cos(t) is two waves i've mixed and sin(t) + cos(t)*0.25 is two waves but one is made quieter, this 0.25 factor is usually called the weight or volume.

API

I like the style that miniaudio (not unique to it) where you just have a node graph and can pipe things like an audio source through reverb and eventually end up in the mixer. Check out section 7 in the docs, line 1989 for more info and some neat diagrams.

Low pass filter

https://dobrian.github.io/cmp/topics/filters/lowpassfilter.html

Future reading

It might be helpful to read the miniaudio docs along with the code which talks about passes, search for process_pcm_frames in that header file. I'd recommend creating a copy of the function you start at and deleting anything to do with the multiple format types it supports, that way it's just showing the data ops which isn't too spooky.

[bigrando420]

Here's everything else I've gathered so far.

https://tek256.com/posts/game-audio/ - excellent overview wrt games

Allen's Wave series
Wave [7]: More Audio Generation - has a great explanation of dbs/volume

nitty gritty DSP stuff

The topic stretches deep and wide, at least as much as computer graphics (whilst regrettably not getting the same sort of love and attention).
If you do decide to go deep, for a general overview https://www.dspguide.com/ is worthwhile. Heavy on the math, but easy to get into and well explained. Convolutions are useful for realistic sounding "impulse response" reverbs, but you'll need to rewrite the naive BASIC examples to use vectors, SSE extensions etc
-Kapsy

http://www.dspguide.com/pdfbook.htm

^ respite from overwhelm

All of this jargon could make the task of writing a mixer seem too difficult for a beginner to attempt.
In reality, games (especially 2D games) can get away with having suuuper unsophisticated mixers.
Personally, I think you should write your own mixer, even if you end up using a library in the end. But first, start with the basics.
Write a program that can load the PCM data from a file (with dr_wav.h or something) and play it through Sokol. Then think to yourself: "What else does my game need?". You'll probably think of a list of things along the lines of:

• Play multiple sounds at once (obviously)
• Allow for variable pitch/panning/volume for each sound
• Allow for things like Play/Pause/Stop/Fade/etc to be called from the game thread
• Maybe add additional DSP effects like reverb/delay/filters

But remember, don't let the DSP stuff scare you.
Kapsy mentioned that you need to filter out aliasing. I find that this is rarely the case in games. Usually, aliasing only becomes noticeable when you try to pitch shift by a really sizeable amount.
In other words, basically do what Casey does, incrementally building your way up. Don't try to solve problems that your game doesn't even have. Also, the basic DSP implementations for delay/reverb aren't even that difficult (I'm not sure about convolution though, the only reverb I know how to implement is basically just delay)
-Andrew Harter

FIRST STEPS

As always... https://guide.handmadehero.org/code/day139/

Jai-specifics:

I too bind to sokol_audio rather than use Sound_Player. I also use stb_vorbis and Wav_File.jai from the standard modules. Wav_File.jai is like 2 lines of code away from supporting WAV_FORMAT_IEEE_FLOAT already, so I use a modified version of that file and then I'm 100% float samples everywhere.