It's a well-known fact that our mobile devices come equipped with all kinds of sensors used for various features that we tend to take for granted. For example, a phone knows if it's held in portrait or landscape orientation, it knows if you're walking in a certain direction, and it can even detect if you've left it on a table screen-up or screen-down. All of this information is calculated internally using various electronic sensors. But, can we make use of these sensors for our games? The answer is yes, and it's relatively straightforward.
In this blog post, you will learn how to use mobile phone sensors to control the transformations of objects in the Godot Engine.
This post will cover the following topics:
- Understanding the concepts of pitch, roll, and yaw in 3D space.
- Using the gyroscope to set the pitch, roll, and yaw.
- Using the accelerometer and magnetometer to set the pitch, roll, and yaw.
- Combining all three sensors for better control.
Throughout the post, we will use these values to control the camera node of the scene.
To follow this tutorial you will need:
- Some knowledge of the Godot Engine
- A mobile phone with gyroscope, magnetometer and accelerometer
- Basic math skills (trigonometry mainly)
- Basic physics knowledge
- Ramatak Mobile Studio or Godot 3.5.1 set up to export to Android
The project required for this tutorial is simple. Just use a Spatial node as the root of your scene, and add a Camera node to it. You should also add some meshes to have as a reference for when we start moving the camera. Your base project should look something like this:
We're going to use sensors to orient the camera in our game. To do this, we need to understand some basic definitions of camera rotation. With the axis convention used in the Godot engine, we can define the following:
- Pitch: rotation arround the x-axis.
- Roll: rotation arround the z-axis.
- Yaw: rotation arround the y_axis.
Our goal is to determine these three parameters using our cellphone sensors.
Let's start by looking at the gyroscope. A gyroscope gives us the angular velocity in radians per second (Input
system. So let's use this to get the pitch for our camera.
Add a script to the camera with the following code:
extends Camera
var pitch: float = 0.0
func _physics_process(delta):
var gyroscope: Vector3 = Input.get_gyroscope()
pitch += gyroscope.z * delta
rotation.z = pitch
Run your project now on your device, and move it around. You will notice that the camera moves accordingly!
So lets complete the previous script for roll and yaw:
extends Camera
var pitch: float = 0.0
var roll: float = 0.0
var yaw: float = 0.0
func _physics_process(delta):
var gyroscope: Vector3 = Input.get_gyroscope()
pitch += gyroscope.x * delta
roll += gyroscope.z * delta
yaw += gyroscope.y * delta
rotation = Vector3(pitch, yaw, roll)
Try it again and see that now you can freely rotate your camera to inspect the scene! You can now control the orientation of your camera using your mobile phone. Great! But we're not done yet.
If you played around enough with the previous code, you will soon note that if the roll is
var gyroscope: Vector3 = Input.get_gyroscope().rotated(-Vector3.FORWARD, roll)
Note: Remember that in Godot Vector3.FORWARD
is a vector pointing in the opposite direction of the z axix.
So, are we done now? Unfortunately, no...
Gyroscopes are cool sensors, but they have a problem: they tend to drift. If you move your camera around a lot and then put your phone in the same orientation it had when the game started, likely, you won't get the same view as when the game launched.
The accelerometer is a sensor that, measures acceleration in units of
Godot provides two measurements with the accelerometer:
Input.get_accelerometer()
: gives you the sensor values in each axis.Input.get_gravity()
: gives you the values but only takes into account gravity.
The latter is better for measuring pitch and roll.
We can use the gravity value to get the pitch and the roll, but we won't be able to get the yaw as the gravity resultant is always parallel to the y-axis.
If you don't care much about the math, just use this code and skip the next section:
func _physics_process(delta):
var gravity: Vector3 = Input.get_gravity()
roll = atan2(-gravity.x, -gravity.y)
gravity = gravity.rotated(-Vector3.FORWARD, rotation.z)
pitch = atan2(gravity.z, -gravity.y)
rotation = Vector3(pitch, yaw, roll)
Trying to not go very deep with math, this picture is provided to help visualize the situation, which assumes that the roll is equal to
The picture shows the local axes of the device when some pitch is applied. By definition, the pitch is the angle between the horizontal and the z-axis (i.e. the rotation around the x-axis). The magnitudes
By inverting atan2(gravity.z, -gravity.y)
.
To account for cases when roll is not equal to gravity = gravity.rotated(-Vector3.FORWARD, rotation.z)
.
The formula for the roll value can be obtained through a similar analysis, and is given by roll = atan2(-gravity.x, -gravity.y)
.
To determine the yaw, we use the magnetometer. This sensor measures the magnetic fields and is capable of detecting the direction of the north pole due to the earth's magnetic field. By considering it as an accelerometer that points towards the north, we can calculate the yaw value in a similar way than we did for roll and pitch:
yaw = atan2(-magnet.x, magnet.z)
However, there are two factors to consider when calculating yaw. First, we need to rotate the readings from the magnetometer to take the roll and pitch into account.
var magnet: Vector3 = Input.get_magnetometer().rotated(-Vector3.FORWARD, rotation.z).rotated(Vector3.RIGHT, rotation.x)
yaw = atan2(-magnet.x, magnet.z)
Secondly, to align the yaw value with the world coordinates in our game, we need to calculate an initial yaw value and subtract it from all subsequent calculations. The resulting script is:
extends Camera
var pitch: float = 0.0
var roll: float = 0.0
var yaw: float = 0.0
var initial_yaw : float = 0.0
func _ready():
yield(get_tree(),"idle_frame")
var magnet: Vector3 = Input.get_magnetometer()
initial_yaw = atan2(-magnet.x, magnet.z)
func _physics_process(delta):
var magnet: Vector3 = Input.get_magnetometer().rotated(-Vector3.FORWARD, rotation.z).rotated(Vector3.RIGHT, rotation.x)
var gravity: Vector3 = Input.get_gravity()
roll = atan2(-gravity.x, -gravity.y)
gravity = gravity.rotated(-Vector3.FORWARD, rotation.z)
pitch = atan2(gravity.z, -gravity.y)
yaw = atan2(-magnet.x, magnet.z)
rotation = Vector3(pitch, yaw - initial_yaw, roll)
Note: you need to wait for a frame before getting the magnetometer reading in _ready()
, otherwise it will return Vector3.ZERO
.
We achieved our camera movement using another approach, but this method is not perfect either. As previously mentioned, gyroscopes have drift issues, but magnetometers and accelerometers aren't a silver bullet either: they have noise. If your sensors are cheap as mine, you will notice the camera orientation is not smooth at all, particularly in regards to yaw.
Gyroscopes have little noise but a lot of drift, and magnetometers and accelerometers have little drift but a lot of noise. Fortunately, mathematical techniques allow us to combine the benefits of both sensors, providing a better solution!
There are several methods for combining the readings from the three sensors to get a more accurate estimation of orientation values. However, for our purposes, the simplest solution will suffice.
We can get roll, pitch, and yaw from both the gyroscope and the accelerometer/magnetometer. By taking a weighted average of these values, we can mitigate the drift from one measure and the noise from the other. For example, to calculate the roll value, we can use the following code:
roll = k (roll + gyroscope.z * delta) + (1-k) * roll_acc
Here, k
is a constant between roll_acc
is the roll value calculated with the accelerometer. A value of k
close to
There is one final challenge: the roll calculated with a gyroscope can increase without bonds, but the one calculated with the accelerometer is limited to the range
Godot comes to the rescue again! The complementary filter is essentially a linear interpolation between the two values, and Godot has a function specifically designed to interpolate between angles while wrapping them correctly. This function is lerp_angle()
and in this case, we can use it like this:
roll = lerp_angle(roll_acc, roll + gyroscope.z * delta, k)
With this in mind, the final code results in:
extends Camera
var pitch: float = 0.0
var roll: float = 0.0
var yaw: float = 0.0
var initial_yaw : float = 0.0
var k : float = 0.98
func _ready():
yield(get_tree(),"idle_frame")
var magnet: Vector3 = Input.get_magnetometer()
print(magnet)
initial_yaw = atan2(-magnet.x, magnet.z)
func _physics_process(delta):
var magnet: Vector3 = Input.get_magnetometer().rotated(-Vector3.FORWARD, rotation.z).rotated(Vector3.RIGHT, rotation.x)
var gravity: Vector3 = Input.get_gravity()
var roll_acc = atan2(-gravity.x, -gravity.y)
gravity = gravity.rotated(-Vector3.FORWARD, rotation.z)
var pitch_acc = atan2(gravity.z, -gravity.y)
var yaw_magnet = atan2(-magnet.x, magnet.z)
var gyroscope: Vector3 = Input.get_gyroscope().rotated(-Vector3.FORWARD, roll)
pitch = lerp_angle(pitch_acc, pitch + gyroscope.x * delta, k)
yaw = lerp_angle(yaw_magnet, yaw + gyroscope.y * delta, k)
roll = lerp_angle(roll_acc, roll + gyroscope.z * delta, k)
rotation = Vector3(pitch, yaw - initial_yaw, roll)
And that's it! With this method, you can now control the Camera
or any other node that inherits from Spatial
using the sensors in your phone. The possibilities for creative game design are endless! Have fun experimenting with this technique.