Systems
randy408 opened this issue · 7 comments
My thoughts so far on how systems should work:
-
ecs.system()
wraps the flecs function like the existing APIs we have so far -
There has to be a dummy
ecs_lua_entrypoint(ecs_iter_t *it)
between the VM and flecs -
When the system is called from flecs it pushes each "argument" as an array of components, then copies it backs into flecs when it's finished
-
flecs-lua probably shouldn't "own" the lua_State, so it has be kept in mind that the VM can persist across frames and:
-
if a system has a
Init()
we definitely have to persist that VM untilDeInit()
anyway
Some things that are not so obvious:
-
If there are multiple systems registered from a state, how do we figure out which Lua system is supposed to run?
-
Do we need a lua_State per system?
-
Can we make multithreading work?
-
How does it interact with host scripts that aren't modules/systems?
-
Can we still make "simple" scripts work where
sleep()
is possible? -
Reloading script/systems is kinda important, what changes in flecs-lua would that require?
What I did so far:
ecs.system()
creates a flecs system that executes the dummy function- Lua systems are recorded in the Lua Registry and are retrieved by the dummy function to call these functions
- Each LuaSystem's index in the registry is passed to the system through
it->param
This kind of guarantees that a LuaSystem imitates a flecs system as much as it could. However, I'm currently trying to find ways to:
- pass an array to Lua without needing to literally copy each element on every system call. Not sure if there is even a way to do that.
- give
ecs_lua_entrypoint(..)
access tolua_State
without making it a global. I tried adding it as a component to each system but this seems to introduce the overhead of callingecs_get(world, system, LuaState)
on each call to the system
Regarding multithreading, I don't think Lua has support for multithreading. As far as I know, we'd need to have multiple lua_State
s for this to be achievable
Passing arrays without copying is premature optimization, I think the bigger problem is that if flecs-lua manages all the de/serialization everything will be a dumb table. It has to be aware which components need special metatables, e.g. vec3 + vec3
or scalar * vec3
should work as expected.
If we solve that efficiently we could make the columns special arrays, initially we could push the elements as-is on access and read back the pushed elements at the end.
There are many ways to do it which depend on the use case, which is why I don't want to get into it at this point, we could check for [in]
for now.
The API needs some changes for multithreading, I'll have something ready soon.
A weird question. Would it be a bit dumb if we deviated a bit from the standard flecs API? What I have in mind is for the systems to work on each entity individually rather than on entire arrays.
So the ecs_lua_entrypoint(..)
would be something like this:
void ecs_lua_entrypoint(ecs_iter_t *it)
{
for(int i = 0; i < it->count; ++i)
{
lua_rawgeti(L, LUA_REGISTRYINDEX, (long)(it->param));
// TODO: Pass stuff
lua_pcall(L, 0, 0, 0);
}
}
instead of this:
void ecs_lua_entrypoint(ecs_iter_t *it)
{
lua_rawgeti(L, LUA_REGISTRYINDEX, (long)(it->param));
// TODO: Pass stuff
lua_pcall(L, 0, 0, 0);
}
This way the lua system will be something like:
function move(position, velocity)
position.x = position.x + velocity.x
position.y = position.y + velocity.y
return position, velocity
end
instead of having to loop in Lua itself.
To make multithreading work flecs-lua could create states per-system and issue a callback for the host to de/initialize the state.
The registry would look like this:
-
Main state
ecs_lua
- wasesc_world
, now a userdata ofecx_lua_ctx
which now also holds the callback func pointersecs_lua_systems
- lightuserdata array ofecs_lua_system
's, we only have access to the main state on deinit so
these structs would contain the pointer to thelua_State
to be cleaned up onecs_lua_exit()
-
System states
ecs_lua
- same as in the main state, but as lightuserdataecs_lua_system
- full userdata struct
-
ecs_lua_init()
doesn't change much aside from taking aecs_lua_ctx*
with more arguments -
ecs_lua_exit()
takes the main state and deinits all systems listed inecs_lua_systems
ecs.system()
CFunction:
- Call host for a new
lua_State
, initializeecs
inside it - Allocate a
ecs_lua_system
struct as full userdata, store it in the registry keyed withecs_lua_system
- Append to
ecs_lua_systems
it->param
is theecs_lua_system*
I'm looking into the module API and how it could interact with scripts.
What I have in mind is for the systems to work on each entity individually rather than on entire arrays.
There are situations where the opposite is preferable or even required, e.g. individual loops over with 2 components out of many one after the other, it's also possible to make API calls inside systems and create additional loops. Because this restricts access to just one cell of each column on each iteration it can't be made the default behavior.
What I have in mind is for the systems to work on each entity individually rather than on entire arrays.
There are situations where the opposite is preferable or even required, e.g. individual loops over with 2 components out of many one after the other, it's also possible to make API calls inside systems and create additional loops. Because this restricts access to just one cell of each column on each iteration it can't be made the default behavior.
Yeah, makes sense. I was just thinking about how that would give a new level of parallelism in the systems, but it does seem to be quite limiting.