/vr-speech-sandbox-vive

A sample application that demonstrates how to integrate voice commands and speech recognition into a virtual reality experience.

Primary LanguageC#Apache License 2.0Apache-2.0

Create voice commands for VR experiences with Watson services

In this developer journey we will create a Virtual Reality game based on Watson's Speech-to-Text and Watson's Conversation services.

In Virtual Reality, where you truly “inhabit” the space, speech can feel like a more natural interface than other methods. Providing speech controls allows developers to create more immersive experiences. The HTC Vive is the 3rd most popular head-mounted VR devices (not including Google Cardboard) and an ideal candidate for Speech interaction, selling roughly 400 thousand units in 2016.

When the reader has completed this journey, they will understand how to:

  • Add IBM Watson Speech-to-Text and Conversation to a Virtual Reality environment build in Unity.

With Watson

Want to take your Watson app to the next level? Looking to leverage Watson Brand assets? Join the With Watson program which provides exclusive brand, marketing, and tech resources to amplify and accelerate your Watson embedded commercial solution.

Included components

Featured technologies

  • Unity: A cross-platform game engine used to develop video games for PC, consoles, mobile devices and websites.

Steps

  1. Before you begin
  2. Create Bluemix services
  3. Building and Running

1. Before You Begin

2. Create Bluemix services

On your local machine:

  1. git clone https://github.com/IBM/vr-speech-sandbox-vive.git
  2. cd vr-speech-sandbox-vive

In Bluemix:

  1. Create a Speech-To-Text service instance.
  2. Create a Conversation service instance.
  3. Once you see the services in the Dashboard, select the Conversation service you created and click the "Launch Tool" button.
  4. After logging into the Conversation Tool, click the "Import" button.
  5. Import the Conversation workspace.json file located in your clone of this repository.

3. Building and Running

If you followed the previous steps you should already be inside your local clone and ready to get started running the app from Unity.

  1. git clone https://github.com/watson-developer-cloud/unity-sdk.git
  2. Open Unity and inside the project launcher select the Open button.
  3. Navigate to where you cloned this repository and open the "Creation Sandbox" directory.
  4. If prompted to upgrade the project to a newer Unity version, do so.
  5. Follow these instructions to add the Watson Unity SDK downloaded in step 1 to the project.
  6. Follow these instructions to add your Speech To Text and Conversation service credentials (located on IBM Bluemix).
  7. Select Advanced Mode in the configuration window.
  8. Click Add Variable and name your new variable ConversationV1_ID then set its value to the Workspace ID of your Conversation workspace. Variable Configuration Example You can find your workspace ID by selecting the expansion menu on your conversation workspace and selecting View details. View Details Location
  9. In the Unity editor project tab, select Assets->Scenes->MainGame->MainMenu and double click to load the scene.
  10. Press Play

Sample output

Links

License

Apache 2.0