Bachelor thesis project "Interactions between Virtual Agents and Smart Objects in Mixed Reality Learning Applications"

The developed application is called ARTutor. It offers a learning experience for acquiring the Estonian language in augmented reality (AR). Upon opening the app, the learner is invited to place virtual objects and label physical objects in their physical environment. Then, a virtual avatar called Mira offers to demonstrate the learner four different interactions on each placed or labeled object. Before and after each interaction, in-app flash cards with the expressions matching Mira’s interactions are presented to the learner. The flash cards are in Estonian with English translations.

Installation

The Unity project is compatible with Unity 2020.3.27f1+. The Xcode build is compatible with Xcode Version 13.4.1+. ARTutor was developed for the 12.9-inch iPad Pro 2021 with LiDAR.

Working with the Unity project

Install Unity Hub and download Unity 2020.3.27f1.
Open the project in Unity and open the Meshing scene from the project files.
Install the Remote Plugin for Unity or replace the scripts MeshClassificationFracking and MeshClassificationManager with theor original versions from arfoundation-demos.

Creating an iOS build

A ready-made build is included as a release in the repository. Unless you would like to create your own build, you can skip this part of the instruction.

If you installed the Remote Plugin, set it up according to the instructions provided by its developer.
In the project settings, select ARKit and ARCode as plug-in providers for iOS and Android respectively under XR Plug-in Management.
In the project settings, change the Player settings to match your Xcode company name, bundle identifier, etc. Besides, add the mandatory Camera Usage Description and Microphone Usage Description. Finally, set the Target minimum iOS Version to 11.0.
In the build settings of the project, select the Meshing scene, choose iOS as the platform, select Xcode Version 13.4.1 or later, and check Development build and Deep profiling support.
Build the project.

Running an iOS build

Open the build in Xcode and adjust your General and Signing & Capabilities settings to match your bundle identifier, team, etc. Besides, select the Automatically manage signing option.
Connect an iPad and select is as a target device in Xcode.
Run the build.

If you have any questions regarding the installation process, contact me via danylo.bekhter@rwth-aachen.de.