Godot Android Streaming Texture (GAST) Plugin
This is a collection of Godot Android plugins that leverage Godot's support for OpenGL external textures to enable rendering and interaction with Android views in a Godot project's scenegraph.
Note: The plugins are only supported on the Godot 3.2.x branch, starting with version 3.2.2.
License
This project is released under the MIT license.
Contributions
The project is in alpha state (base functionality is available but API is subject to change). Contributions are welcomed!
Setup
Clone the repository and initialize the submodules with git submodule update --init --recursive
.
- Note: When you've pulled a newer version, make sure to run
git submodule update --init --recursive
.
Godot CPP library
- The
git submodule update --init --recursive
command should have checked out the godot-cpp repo under thecore/libs/godot-cpp
directory. - Navigate to the
core/libs/godot-cpp
directory and follow these steps:- Generate the
godot-cpp
bindings and static libraries with the following commands:scons platform=android generate_bindings=yes android_arch=arm64v8 target=release ANDROID_NDK_ROOT="/PATH-TO-ANDROID-NDK/"
scons platform=android generate_bindings=yes android_arch=arm64v8 target=debug ANDROID_NDK_ROOT="/PATH-TO-ANDROID-NDK/"
- Generate the
Note: If you run into issues, check out the repo setup instructions.
IDE
The project uses Android Studio for development. Make sure you have Android Studio version 4.0 or higher and open the project within Android Studio.
GAST Plugins
Architecture
The GAST plugin is written in C++ and Kotlin.
The Kotlin layer leverages the Godot Android Plugin framework to integrate with a Godot Android application. It provides the public APIs for Android clients to access the plugin core functionality in order to render, interact with and manipulate content generated by the Android View system in Godot’s scenegrah.
The core functionality is written in C++. It leverages the Godot GDNative C++ bindings in order to access and use the GDNative API. Through that API, it’s able to access at runtime the project scenegraph and manipulate it by adding, updating and/or removing Godot nodes and/or resources.
GastNode
This is the element in the scenegraph responsible for rendering and supporting interaction with an Android view.
Structure
A GastNode is made of the following Godot nodes and resources:
- StaticBody node as the root node, which allows the GastNode to detect collision events (if enabled by the user).
- A StaticBody node requires a Shape resource to define the object’s collision bounds. For the GastNode, we’re using a CollisionShape node who's shape is set whenever the child's mesh is updated.
- Since our GastNode is ultimately a mesh in 3D space, we make use of Godot’s MeshInstance node to render the GastNode.
Once a GastNode is created, the client gains the ability to retrieve a Surface instance via the GastNode#bindSurface() API. The Surface instance can be used as an output destination onto which Android Views, images or videos can be rendered. This is the process used by the GAST-Video and GAST-WebView plugins.
Input Handling
Using the GastInputListener interface via the GastManager#registerGastInputListener(...) and GastManager#unregisterGastInputListener(...) methods, the GastManager instance relays interaction events from the generated node(s) to the client. This provides the client with enough information to generate and feed Android MotionEvent events to the Android view.
Two types of events are supported:
Input Events
The plugin uses the Godot Input API to subscribe to dispatched input action events (e.g: button press). It listens for the events registered by the client via the GastInputListener#getInputActionsToMonitor() method, and notifies the client accordingly via the GastInputListener#onMainInputAction(...) callback.
Collision Events
Godot’s Physics API is used in order to detect collision events between the GastNode and a ray cast from the input pointer (in VR the input pointer consists of a tracked controller/hand). This is used to provide support for detecting click, hover and scroll events targeted at the GastNode.
These events are dispatched to the GDScript code via the following signals:
hover_input_event
for hover eventspress_input_event
for click press eventsrelease_input_event
for click release eventsscroll_input_event
for scroll events
Similarly, the Android code can listen to these events using the corresponding methods:
- GastInputListener#onMainInputHover(...) for hover events
- GastInputListener#onMainInputPress(...) for click press events
- GastInputListener#onMainInputRelease(...) for click release events
- GastInputListener#onMainInputScroll(...) for scroll events
The dispatched events include the nodepath of the targeted Gast node, the nodepath of the colliding ray cast and information specific to the type of the event (e.g: hover location for a hover event).
Example code:
func _ready():
...
var gast_loader = load("res://godot/plugin/v1/gast/GastLoader.gdns").new()
gast_loader.initialize()
gast_loader.connect("hover_input_event", self, "_on_gast_hover_input_event")
gast_loader.connect("press_input_event", self, "_on_gast_press_input_event")
gast_loader.connect("release_input_event", self, "_on_gast_release_input_event")
gast_loader.connect("scroll_input_event", self, "_on_gast_scroll_input_event")
...
func _on_gast_hover_input_event(node_path: String, event_origin_id: String, x_percent: float, y_percent: float):
pass
func _on_gast_press_input_event(node_path: String, event_origin_id: String, x_percent: float, y_percent: float):
pass
func _on_gast_release_input_event(node_path: String, event_origin_id: String, x_percent: float, y_percent: float):
pass
func _on_gast_scroll_input_event(node_path: String, event_origin_id: String, x_percent: float, y_percent: float, horizontal_delta: float, vertical_delta: float):
pass
Collision Events Setup
A couple of requirements must be followed for the collision events handling to be properly set up.
- The colliding raycast must belong to the gast_ray_caster node group. This allows to filter unwanted raycast collisions.
- Since click and scroll events rely on additional input events (e.g: button press, joystick
tilt), the plugin monitors a set of input action events
with the characteristics listed below. It’s the responsibility of the client to declare
and register the corresponding input actions
in the Godot project.
- The monitored input action prefix is generated by replacing the slash character (“/”)
in the colliding raycast’s nodepath
with the underscore character (“_”).
- E.g: input action prefix for a RayCast node with path /root/Main/Controller/RayCast would be _root_Main_Controller_RayCast
- There is one input action per raycast for click events. It’s generated by appending the
“_click” suffix to the input action prefix generated as described above.
- E.g: click action event for a RayCast node with path /root/Main/Controller/RayCast would be _root_Main_Controller_RayCast_click
- There are two input actions per raycast for horizontal scroll events. They are generated by
appending “_left_scroll” for the left scroll events and “_right_scroll” for the right
scroll events.
- E.g: For a RayCast node with path /root/Main/Controller/RayCast
- Left scroll => _root_Main_Controller_RayCast_left_scroll
- Right scroll => _root_Main_Controller_RayCast_right_scroll
- E.g: For a RayCast node with path /root/Main/Controller/RayCast
- There are two input actions per raycast for vertical scroll events. They are generated by
appending “_up_scroll” for the up scroll events and “_down_scroll” for the down scroll
events.
- E.g: For a RayCast node
with path /root/Main/Controller/RayCast
- Up scroll => _root_Main_Controller_RayCast_up_scroll
- Down scroll => _root_Main_Controller_RayCast_down_scroll
- E.g: For a RayCast node
with path /root/Main/Controller/RayCast
- The monitored input action prefix is generated by replacing the slash character (“/”)
in the colliding raycast’s nodepath
with the underscore character (“_”).
Performance Considerations
The GAST plugin attempts to recycle GastNodes as much as possible, only creating new ones if the pool is empty or exhausted. This allows to keep a lid on the number of generated OpenGL external textures.