touilleMan/godot-python

Need efficient access to PoolIntArray, PoolVector3Array, etc.

garyo opened this issue · 5 comments

garyo commented

For creating/manipulating/exporting meshes, it's important to have efficient access to the Pool*Arrays. It's not trivial, because they are locked to prevent multithreading issues. Godot-python would need to lock the array and get the raw pointer using array.read() to get the lock, reader.ptr() to get the raw data pointer (which can be turned into a python buffer, numpy array etc.), keeping the array.read() object around until done. Unlocking would have to be explicit in python since objects don't always get deleted until gc, something like a python-level reader.free() that would delete the underlying gdnative object.

touilleMan commented

I'm not sure this is something present in the current GDnative API.
Obviously if not we can propose an evolution on the API :-)

Regarding the syntax to get the lock from Python, I would recommend using the with syntax with a context manager:

with my_array.lock:
    ptr = my_array.read()
    ...
garyo commented

I like the context manager approach. Actually I believe everything needed is in GDnative. From gdnative/pool_arrays.h:

godot_pool_vector3_array_read_access GDAPI *godot_pool_vector3_array_read(const godot_pool_vector3_array *p_self);
const godot_vector3 GDAPI *godot_pool_vector3_array_read_access_ptr(const godot_pool_vector3_array_read_access *p_read);
void GDAPI godot_pool_vector3_array_read_access_destroy(godot_pool_vector3_array_read_access *p_read);

What do you think? Is that enough, or do we need more?

touilleMan commented

What do you think? Is that enough, or do we need more?

I never used those api function (is it the ones added in gdnative 1.1 ?), but I guess it should be good enough to do something like:

with my_array.raw_access() as my_array_raw_ptr:
    # Note I don't know the numpy api...
    my_array_numpy = numpy.new_array_from_pointer(my_array_raw_ptr)
    ...
garyo commented

Thomas Herzog added those functions to Godot on 2017-11-03 (in af5c6be074). That commit is in 3.0-stable but I'm not sure about which gdnative release that corresponds to.
As to your suggestion of implementing the raw_access() method using those API funcs -- I'll give that a try next time I need it.

In fact it can be useful even without numpy, using the standard python buffer interface; see https://docs.python.org/3.6/c-api/buffer.html and https://docs.python.org/3.6/c-api/memoryview.html#memoryview-objects. This could be used, for instance, to directly write or read a PoolArray to/from a file.

touilleMan commented

I had a look a this, and it appeared to be really easy to implement, so here it is:
e9347d6 😄

  • 🎉1