In order to come up with a new 3D multimodal interaction paradigm of Natural user interface for virtual and augmented reality application, we built an Multimodal Virtual Reality educational game, in which a user or a learner can navigate or select and manipulate objects in three dimensional space. This immersive experience afforded by the 3D technology gives a learner unprecedented personal control over the learning environment. AlphaLearn use PC and KinectV2.
AlphaLearn is a game that helps kids to learn how to write the pronounced word with boxes containing alphabetic characters. The kids will not use keyboard or mouse to reorder the alphabetic characters, they will use their voice and hand gestures to control boxes.
Befor we start, This game require Kinect V2 device to track the user hand and head movement. The kinect of XBOX one also can be used, since its concedered as kinect V2. to use kinect with PC u need to have Microsoft Kinect V2 Adapter for Windows, which its require USB 3.0 controller.
-
Download and install Kinect SDK 2.0.
-
Download and install Kinect Runtime 2.2.1811.
-
Clone or download this repository.
git clone https://github.com/rahou0/AlphaLearn.git-
Open Unity Editor and upload the game.
-
Download and import the package Kinect v2 Unity Plugin onto the project
The learner listens to the sound that speaks the word and tries to form that word with cubes with alphabetic characters in their faces Scattered in the ground. The user can use several modalities to interact with the virtual learning environment such as hand gestures or speech for select and manipulate the cubes, as well as using his head or hand for navigating throw virtual space. For more Details, please read the Documentation
.png?raw=true)
.png?raw=true)
.png?raw=true)
Distributed under the MIT License. See LICENSE for more information.
Abderahim Hamani - @twitter
Project Link: https://github.com/rahou0/AlphaLearn