/mxchip-iot-devkit-translator

Use IoT DevKit AZ3166 with Azure Function and Cognitive Services to make a language translator

Primary LanguageC#MIT LicenseMIT

page_type languages products name description urlFragment
sample
c
cpp
azure
azure-iot-hub
Azure MXChip IoT DevKit Translator
Turn DevKit as a translator by using Bing Speech Translator service.
sample

Use IoT DevKit AZ3166 with Azure Functions and Cognitive Services to make a language translator

In this article, you learn how to make IoT DevKit as a language translator by using Azure Cognitive Services. It records your voice and translates it to English text shown on the DevKit screen.

The MXChip IoT DevKit is an all-in-one Arduino compatible board with rich peripherals and sensors. You can develop for it using Azure IoT Tools extension pack in Visual Studio Code. The projects catalog contains sample applications to help you prototype IoT solutions.

Before you begin

To complete the steps in this tutorial, first do the following tasks:

Create Azure Cognitive Service

  1. In the Azure portal, click Create a resource and search for Speech. Fill out the form to create the Speech Service. Speech service

  2. Go to the Speech service you just created, click Keys section to copy and note down the Key1 for DevKit accessing to it. Copy keys

Open sample project

  1. Make sure your IoT DevKit is not connected to your computer. Start VS Code first, and then connect the DevKit to your computer.

  2. Click F1 to open the command palette, type and select Azure IoT Device Workbench: Open Examples.... Then select IoT DevKit as board.

  3. In the IoT Workbench Examples page, find DevKit Translator and click Open Sample. Then selects the default path to download the sample code. Open sample

Use Speech Service with Azure Functions

  1. In VS Code, click F1, type and select Azure IoT Device Workbench: Provision Azure Services.... Provision Azure services

  2. Follow the steps to finish provisioning of Azure IoT Hub and Azure Functions. Provision steps

    Take a note of the Azure IoT Hub device name you created.

  3. Open Functions\DevKitTranslatorFunction.cs and update the following lines of code with the device name and Speech Service key you noted down.

    // Subscription Key of Speech Service
const string speechSubscriptionKey = "";

// Region of the speech service, see https://docs.microsoft.com/azure/cognitive-services/speech-service/regions for more details.
const string speechServiceRegion = "";

// Device ID
const string deviceName = "";

  4. Click F1, type and select Azure IoT Device Workbench: Deploy to Azure.... If VS Code asks for confirmation for redeployment, click Yes. Deploy warning

  5. Make sure the deployment is successful. Deploy success

  6. In Azure portal, go to Functions Apps section, find the Azure Function app just created. Click devkit_translator, then click </> Get Function URL to copy the URL. Copy function URL

  7. Paste the URL into azure_config.h file. Azure config

    [NOTE] If the Function app does not work properly, check this FAQ section to resolve it.

Build and upload device code

  1. Switch the DevKit to configuration mode by:

    • Hold down button A.
    • Press and release Reset button.

    You will see the screen displays the DevKit ID and Configuration.

    DevKit configuration mode

  2. Click F1, type and select Azure IoT Device Workbench: Configure Device Settings... > Config Device Connection String. Select Select IoT Hub Device Connection String to configure it to the DevKit. Configure connection string

  3. You will see the notification once it's done successfully. Configure connection string success

  4. Click F1 again, type and select Azure IoT Device Workbench: Upload Device Code. It starts compile and upload the code to DevKit. Device upload

Test the project

After app initialization, follow the instructions on the DevKit screen. The default source language is Chinese.

To select another language for translation:

  1. Press button A to enter setup mode.

  2. Press button B to scroll all supported source languages.

  3. Press button A to confirm your choice of source language.

  4. Press and hold button B while speaking, then release button B to initiate the translation.

  5. The translated text in English shows on the screen.

Scroll to select language

Translation result

On the translation result screen, you can:

  • Press buttons A and B to scroll and select the source language.

  • Press the B button to talk. To send the voice and get the translation text, release the B button.

How it works

mini-solution-voice-to-tweet-diagram

The IoT DevKit records your voice then posts an HTTP request to trigger Azure Functions. Azure Functions calls the cognitive service speech translator API to do the translation. After Azure Functions gets the translation text, it sends a C2D message to the device. Then the translation is displayed on the screen.

Problems and feedback

If you encounter problems, refer to the IoT DevKit FAQ or reach out to us using the following channels:

Next steps

You have learned how to use the IoT DevKit as a translator by using Azure Functions and Cognitive Services. In this how-to, you learned how to:

  • Use Visual Studio Code task to automate cloud provisions
  • Configure Azure IoT device connection string
  • Deploy the Azure Function
  • Test the voice message translation

Advance to the other tutorials to learn:

Connect IoT DevKit AZ3166 to Azure IoT Remote Monitoring solution accelerator