Bololens

Summary

Bot + Hololens = Bololens

This repo provides a nice and easy integration of Chat Bot Clients in Unity application targeting UWP platforms. It is mainly developed for Hololens integration.

As bots and Conversation as a Platform are addressing a new way of interacting through Natural UI, it sounds obvious that bots could have a Digital Identity and Representation. What could be better than Mixed Reality to address the issue?

As we are working in Unity you can find the main project under the BoUnity folder (Bot + Unity = ...)

The only currently supported bot framework is the Microsoft Bot framework with the know following Limitations.

TL;DR

Import the entire content from the bololens.unitypackage from the Dist folder in your Unity application.
Drag the Bololens Prefab from the Bololens/Prefabs folder in your scene.
Customize the Url or Token from the BotBrain component to either your Bot Token or a service emitting tokens.
Customize your action keywords like Activation and Desactivation in the bot brain component.
Run in the editor. As Text To Speech and Speech To Text are not available in the editor, you can use the Embedded GUI to interac with the bot.
Deploy in Hololens. Use Voice Commands through the configured Activation keywords to start discussing with your bot.
Voilà

Base Principles and caracteristics

As the bot personnification is trying to mimic Natural User interactions, His caracteristics should be close from humans once. The bot has then be splitted in a bunch of isolated reusable and customizable sections all available through their respective Unity Components:

Brain or Intelligence: The orchestrator and main compute of all the other caracteristics.
Materialisation: The Digital body of the bot containing all the visualisations and animations.
Hearing: The component responsible of listening for inputs (Voice or GUI).
Speech: This gives our bot a voice !
Sight: The bot can now see your world as well as holograms but still no XRay vision.
Personality: Depending on the messages exchanged, the bot feelings can evolve. A few Personalities are built in (Normal, Crazy...)
Memory: All your discussions might now have a long term impact.
Networking: Remote Service Connectivity providing web access to the bot.

Going Further

Conclusion