/vrTrainingFingerAlphabet

VR Training Application for Finger Alphabet from the German Sign Language

Primary LanguageC#GNU General Public License v3.0GPL-3.0

VR-Trainingsapplikation-App für Fingeralphabet

VR-Trainingsapplikation-App für Fingeralphabet Thumbnail

IPA-Abschlussarbeit von Cédric Girardin

License: GPL v3

Overview

This is the final project (IPA) of our apprentice Cédric Girardin at the computer science department of the Berne University of Applied Science.
The VR-Application runs on a Oculus-Quest and uses its Hand tracking feature. The Application can validate up to 23 Hand signs from the German Alphabet.

This Project was made in 2 weeks/ 10 Work days: 30.03.2020-08.05.2020 (Yes, the documentation was also made within does 2 weeks).

The Readme file explains shortly folowing topics:

  • Setup
  • Documentation
  • Plugins
  • Handform validation
  • Handdata
  • Domainmodel

Setup

  1. Clone the unity project
git clone https://github.com/cpvrlab/vrTrainingFingerAlphabet.git
  1. Open Unity Hub->Add, Select the Project where its cloned
  2. Select Unity Version (2019.3.06f) and open the project

Documentation

The Documentation is in German and uses the HERMES 5.1 Project method.

└───Assets
│   └───Documents
│       │   IPA_CédricGirardin.pdf

Plugins

The Unity Project uses folowing plugins:

Name Link
ui_ux_in_vr_2019_2 https://github.com/soma1294/ui_ux_in_vr_2019_2
Oculus Integration https://assetstore.unity.com/packages/tools/integration/oculus-integration-82022
JSON .NET For Unity https://assetstore.unity.com/packages/tools/input-management/json-net-for-unity-11347#description
Monumental Journey https://www.youtube.com/audiolibrary/music?nv=1
All Sound Effects https://bigsoundbank.com/

Handform validation

To validate a Handform, it loads a saved hand into the scene, and compares it values with the current hand.
The Validation compares folowing values:

Handsubdivision Value Type BoneIds
Handorientation Vector.up Hand_WristRoot
Fingerangle localEulerAngles Z Intermediate BonesId’s and distal BoneId’s
Fingertipdistances Diffrence of Vector.position Hand_tipId’s

The diffrences then gets weighted and it determents if its right or wrong. This process can be observed in realtime in the insight scene (insight the oculus quest).\

Handdata

In the insight scene is a save button. When pressed it creates a Handformstruct:
structimage

This Handformstruct will then be saved in the handdata.json file.

The Jsonfile will be converted to the values of a handformstruct...

Domainmodel

structimage