Task Scheduler

[katk0smos]

So this is more of a general feature request/enhancement (also a question), but something that would be extremely useful in my use cases would be having a task scheduler so that scripts could, for example, yield back to the engine for a specified duration (like a sleep function). This could also potentially be used for calling callback functions bound to specific events through functions, rather than having specific hook functions.

An example of this might look like this:

-- another benefit of this is having multiple callbacks defined per script
physics_stepped_event:bind(function()
  -- do something
  sleep(5) -- sleep for 5 seconds
  -- do something else
end)

Rather than the current hook-based system:

-- it's a hook so it can only be defined once per script
function physics_stepped_event()
  -- do something
  -- no idea how you'd wait here, maybe call a function and pass the name of another hook to call as well as the time to sleep?
end

I do know this isn't currently possible with the current language implementations, but could something like this be implemented on top of the existing traits? I do know the current system probably works better under Bevy's ECS system, but for me the benefits from having this outweighs the extra work I'd have to do to create the necessary APIs.

It's something that would be extremely useful for my projects, and would be something I would consider implementing myself as well (and of course I'd be willing to consider submitting a PR if you think it'd be a worthy addition). I just don't know enough about how the code is structured to know if I could implement this on top of it or not.

[makspll]

Hi @katk0smos, good to see you again!

Let me see if I understand your use case so I can give you a good answer. So you are in need of:

1. the ability to have multiple functions happen on a single callback
2. the ability to have your script surrender control to the host for a time

Now addressing both:

1. this is already possible If I understand you correctly, you are free to do:

function physics_stepped_event(args)
 physics_stepped_event_handler_1(args);
 physics_stepped_event_handler_2(args);
end

2. Is a bit tricky, the way script hosts work currently is:

• Load and initialise scripts, these get added to the pool of scripts which are to listen to events
• script host handler stages are registered in the bevy stage pipeline
• when the handler stage is run, it picks up on all the events with its priority range and sends them to all the scripts which are subscribed to that kind of event
• All of the scripts run and the stage is over

You see, sleeping functionality would require that the scripts return some sort of message (which is ok), then the host puts those aside, executes the other scripts and then runs those other scripts. This is problematic since any delay here would extend your frame time insanely, and wreck havoc.

I think that adding sleeping as a mechanism is likely not a great solution to this problem, do not despair as there is a good alternative, namely coroutines:

local my_routine;

function on_update()

    if my_routine == nil then
        my_routine = coroutine.create(function()
            local starttime = os.time()
            local endtime = starttime + 5
            while os.time() < endtime do
                print(os.date("waiting %H:%M:%S", os.time()))
                coroutine.yield()
            end

            print(os.date("finished! %H:%M:%S", os.time()))
        end)
    else
        if coroutine.status(my_routine) ~= "dead" then
            coroutine.resume(my_routine)
        else
            print("Couroutine has finished, no longer running")
        end
    end

end

This is normally how you'd handle any sort of delay or timing issue inside a game loop, coroutines let you yield execution untill the next function call, which If you're doing physics will likely happen every 33.33 ms (assuming you're running your on_update hooks at 30Hz) So unless you require nanosecond precision will work perfectly fine.

I am not sure you could build sleep() commands into lua via mlua easilly, but if you really wanted you could definitely write a ScriptHost which runs scripts entirely differently, and without any event loops btw, none of this is enforced upon script hosts, for example the lua one registers all of the machinery itself:

   fn register_with_app(app: &mut App, stage: impl StageLabel) {
        app.add_priority_event::<Self::ScriptEvent>()
            .add_asset::<LuaFile>()
            .init_asset_loader::<LuaLoader>()
            .init_resource::<CachedScriptState<Self>>()
            .init_resource::<ScriptContexts<Self::ScriptContext>>()
            .init_resource::<APIProviders<Self>>()
            .register_type::<ScriptCollection<Self::ScriptAsset>>()
            .register_type::<Script<Self::ScriptAsset>>()
            .register_type::<Handle<LuaFile>>()
            .add_system_set_to_stage(
                stage,
                SystemSet::new()
                    // handle script insertions removal first
                    // then update their contexts later on script asset changes
                    .with_system(
                        script_add_synchronizer::<Self>.before(script_remove_synchronizer::<Self>),
                    )
                    .with_system(
                        script_remove_synchronizer::<Self>
                            .before(script_hot_reload_handler::<Self>),
                    )
                    .with_system(script_hot_reload_handler::<Self>),
            );
    }

meaning you could replace any and all systems you like

[katk0smos]

I submitted a PR (#34) that allows for asynchronous functions, which would theoretically allow the scripts to yield in the background. I believe this is required to implement the task scheduler properly. The only issue is that the values returned by mlua are locked to the lifetime of the Lua struct, and cannot be passed between threads.

I don't know if you happen to know a solution to this issue, or if bevy just isn't a suitable choice for what I'm trying to do. Essentially I need a way of storing all of the Lua contexts and return values on a single thread. I did have the idea of just spawning a background thread that would own everything, but passing messages back and forth seems somewhat messy and I feel like it would defeat the purpose of this crate.

[makspll]

@katk0smos, are you trying to keep a hold of and pass around mlua threads? The way Bevy works, resources/components must be 'static i.e. have no lifetimes attached.

Currently all the scripts added via this plugin are stored in ScriptContexts inside of Mutexes. I am not sure what you mean by storing all of them on a single thread, when the Bevy game loop runs, systems are parallelized and run in their own threads, for this plugin, because the handler systems are exclusive, they have sole control over the contexts when they run, i.e. all of these contexts are controlled by a single thread and events are handled in sequence. The plugin does not dictate what the ScriptHost's do with the events, and the hosts can do whatever they want in the handle_events hook.

I am guessing you're trying to get Thread as a return value from the scripts? You won't be able to store that in a resource or anything, but I believe you should be fine manipulating those threads within the context of a single handle_events hook (although I am not very sure, I've not tried anything like this).

But then I am just guessing, it's a bit hard for me to imagine what you're trying to do (some examples would help), may I ask why coroutines don't suit your use case?

[katk0smos]

I think Bevy just may not be designed for what I'm attempting to do. I was looking to create a Lua API similar to that used in Roblox,

Since I don't believe Bevy will work for what I'm attempting to do with this, I'm just going to close this issue. Thank you for all the help!

[makspll]

@katk0smos looking at the roblox task scheduling, I'd say you'd be able to get the delay mechanics working, i.e. if all you want to do is yield an event call, put it in some queue then immediately after all normal events are handled handle that queue, that should be possible:
say you have:

function EventA()
     return my_function_or_coruotine
end

1. EventA() is called
2. returned function is put on queue of "delayed" events
3. once all normal events are handled, call all of the delayed functions/coroutines untill the queue is empty

This sort of behaviour should be possible entirely within the handle_events hook, anything that requires tasks to be delayed through another frame I would just handle with coroutines as I've shown above.

I am happy to help further if you need (I'll have a deeper look at this sort of use case in a couple days, just haven't been programming lately)

But all the best, hope you find a solution!