AmbientRun/Ambient

Expose full complexity of audio graph in API

philpax opened this issue · 3 comments

The basic audio API we'll expose in 0.1 can only play sounds and optionally repeat them. It can't do anything fancy with them, like attenuating them or filtering them or what have you.

However, our host code offers this functionality. We just need to wire this up in both the host and the guest.

@ten3roberts's suggested approach is to use a graph on the guest side (e.g. petgraph) to construct a graph of nodes, serialize these to a list of nodes and edges, and then send them to the host to reconstruct a Source from. This should be doable, I think.

Our basic example should consist of:

  • A camera looking at a cube.
  • The cube moves around with a looping sound attached to it.
  • You can hear the sound moving with the cube.

This example is very similar to the example we have on the host side. Note that we will need to expose the relevant functionality to do this.

Just talked about this with @ten3roberts.

Seems pretty doable:

  • Duplicate the host Sources over to the guest
  • Remove all the logic for actually sampling
  • Build a list of corresponding WIT-structure nodes and a list of edges
  • Send over to host
  • Host then builds the corresponding tree of Sources

There's some other stuff we'd have to do around wiring up audio_emitter and audio_listener (can just set up some extra components + systems for this), but this should be doable otherwise.

We have some basic audio API now: #304

But there are things to consider:

  • API for handling the loading latency