(documentation/example) FMOD callbacks to bevy

[passivedragon]

One of the more advanced features of FMOD allows pulling information back out of FMOD at runtime.
I've been trying to figure out how to do so with bevy_fmod recently. While I can register callbacks, capturing bevy context to trigger Events for example has been proving... a bit more challenging.

It seems useful enough to me to consider adding an example for such useage (if possible at all).

Has anyone attempted to do this before with bevy? I'll share some code snippets as examples for what I mean.

basic callback example:

use bevy::prelude::*;
use bevy_fmod::fmod_studio::FmodStudio;
use libfmod::ffi::{FMOD_OK, FMOD_RESULT, FMOD_STUDIO_EVENTINSTANCE, FMOD_STUDIO_EVENT_CALLBACK_TYPE};

// Implement a function with the expected FFI signature that calls the closure
extern "C" fn callback_bridge<F>(
    param1: u32,
    param2: *mut FMOD_STUDIO_EVENTINSTANCE,
    param3: *mut std::ffi::c_void,
) -> i32
where
    F: FnMut(u32, *mut FMOD_STUDIO_EVENTINSTANCE, *mut std::ffi::c_void) -> i32,
{
    unsafe {
        // Get the closure from the context data
        let callback_wrapper: &mut CallbackWrapper<F> = &mut *(param3 as *mut CallbackWrapper<F>);

        // Call the closure with the provided parameters
        (callback_wrapper.callback)(param1, param2, param3)
    }
}

impl<F> CallbackWrapper<F>
where
    F: FnMut(u32, *mut FMOD_STUDIO_EVENTINSTANCE, *mut std::ffi::c_void) -> i32,
{
    // Function to create a new CallbackWrapper
    fn new(callback: F) -> CallbackWrapper<F> {
        CallbackWrapper { callback }
    }

    // Function to get the function pointer to the bridge function
    fn get_callback_bridge(&self) -> libfmod::ffi::FMOD_STUDIO_EVENT_CALLBACK {
        Some(callback_bridge::<F>)
    }
}

pub fn setup_fmod_callbacks_system(
    studio: Res<FmodStudio>
) {
   let mut my_closure = move |param1: u32,
                               param2: *mut FMOD_STUDIO_EVENTINSTANCE,
                               param3: *mut std::ffi::c_void| {
        print!("triggered by FMOD event");
        FMOD_OK
    };

    let callback_wrapper = CallbackWrapper::new(my_closure);
    let callback_bridge = callback_wrapper.get_callback_bridge();

    let _ = studio
        .0
        .get_event("event:/my-event")
        .unwrap()
        .set_callback(callback_bridge, 0xFFFF_FFFFu32);
}

This reliably triggers when the appropriate FMOD event triggers, multiple times due to the catch_all mask FFFF_FFFF.

But as for relaying this information to bevy... I've tried an approach using crossbeam-channel.

[passivedragon]

My bevy attempts so far ranged from bevy EventWriters constructions such as NonSend<EventWriter<FmodEvent>>, up to my most recent approach that compiles but unfortunately segfaults at runtime.

#[derive(Resource)]
struct FmodReceiver(Receiver<LevelEvent>);

#[derive(Event)]
pub struct FmodEvent;


pub fn setup_fmod_callbacks_system(
    studio: Res<FmodStudio>
) {
    let (sender, receiver): (Sender<FmodEvent>, Receiver<FmodEvent>) = unbounded();

    commands.insert_resource(FmodReceiver(receiver));
    
   let mut my_closure = move |param1: u32,
                               param2: *mut FMOD_STUDIO_EVENTINSTANCE,
                               param3: *mut std::ffi::c_void| {
        print!("triggered by FMOD event");
        // Send the event to the main thread
        sender.send(FmodEvent::new()).expect("crossbeams sender failed"); // doesn't print, just segfaults
        FMOD_OK
    };

    let callback_wrapper = CallbackWrapper::new(my_closure);
    let callback_bridge = callback_wrapper.get_callback_bridge();

    let _ = studio
        .0
        .get_event("event:/my-event")
        .unwrap()
        .set_callback(callback_bridge, 0xFFFF_FFFFu32);
}

[Salzian]

I'll take a look at this over the weekend.

[passivedragon]

Expanding on this and wishing out loud, I'd love to have bevy_fmod register callbacks on its own and then publish events as bevy events, to make reacting to them in bevy systems that much simpler.

I'll take another look and attempt at this, as I somewhat need this for a usecase of mine.

[passivedragon]

I think I found a solution, I'll see if I can get it to work and create a PR when ready.

[passivedragon]

My solution attempt basically follows the way https://github.com/bevyengine/bevy/blob/main/examples/async_tasks/external_source_external_thread.rs does it, with a channel.
A callback is registered with FMOD, the channel sender is moved and the receiver attached to a bevy PreUpdate system that then receives information from FMOD and publishes bevy events.

I'm a bit lost on which events and how to propagate this way, since there's no simple "attach to absolutely everything" callback. For event information it seems like this kind of thing would need to happen for each event as it is created on the bevy side. It could absolutely be done, just seems cumbersome.

[passivedragon]

Using user_data should make things a bit more simple, enabling using a single callback for multiple events for example. Still this can get very ugly very quickly xD

[passivedragon]

I have a working setup for System events and a solution that would require manual calls for each event that is created.
Not sure how to smartly generalize it for events as is.

[passivedragon]

I've had success with my attempts, but have been unable to come up with a decent UX.

Registering system events via a channel is fairly trivial, but for specific events it gets trickier, since a callback function requires some sort of distinction per event. So far I've simply created functions and then leaked them after registering, which is fine for the system callback since that is only registered once (as long as only one bevy plugin is used etc.), but that's definitely not fine for events as a whole.
A Resource approach might work, storing the callback functions in a NonSend<> and releasing them when not needed anymore might work, but I haven't yet figured out how to do that properly, or even when exactly it would be fine to release and forget.