Voice activated micro:bit

A demo application that makes your micro:bit (2020 model) respond to your voice, built with Edge Impulse. This demo uses Machine Learning to analyze the audio feed coming from the microphone, then showing a smiley on screen when it hears "microbit".

Video tutorial:

How to build (local)

Install CMake, Python 2.7 and the GNU ARM Embedded Toolchain 9. Make sure arm-none-eabi-gcc is in your PATH.

Note: This project currently does not work with GNU ARM Embedded Toolchain 10 (bug)!

Clone this repository:

$ git clone https://github.com/edgeimpulse/voice-activated-microbit

Build the project:
```
$ python build.py
```
And flash the binary to your micro:bit, by dragging MICROBIT.hex onto the MICROBIT disk drive.

How to build (Docker)

Build the container:
```
$ docker build -t microbit_ei_build .
```

Build the project:

macOS, Linux

$ docker run --rm -v $PWD:/data microbit_ei_build

Windows

$ docker run --rm -v "%cd%":/data microbit_ei_build

And flash the binary to your micro:bit, by dragging MICROBIT.hex onto the MICROBIT disk drive.

Viewing the Machine Learning model

The ML model that powers this project is available on Edge Impulse: Micro:bit LIVE 2020.

How to change the keyword

You can build new models using Edge Impulse.

Sign up for an account and open your project.
Download the base dataset - this contains both 'noise' and 'unknown' data that you can use.
Go to Data acquisition, and click the 'Upload' icon.
Choose all the WAV items in the dataset and leave all other settings as-is. Click Upload.
Go to Devices and add your mobile phone.
Go back to Data acquisition and now record your new keyword many times using your phone at frequency 11000Hz.
After uploading click the three dots, select Split sample and click Split to slice your data in 1 second chunks.
Follow these steps to train your model.

Make sure to set the frequency to 11000:

Once you've trained a model go to Deployment, and select C++ Library. Then:

Remove source/edge-impulse-sdk, source/model-parameters and source/tflite-model.
Drag the content of the ZIP file into the source folder.
If you've picked a different keyword, change this in source/MicrophoneInferenceTest.cpp.

Note: the name used for the label of the training-set should correspond exactly to the #define INFERENCING_KEYWORD
Rebuild your application.
Your micro:bit now responds to your own keyword 🚀.

Poor performance due to unbalanced dataset?

Is your model not working properly? Then this is probably due to dataset imbalance (a lot more unknown / noise data compared to your keyword) in combination with our moving average code to reduce false positives.

When running in continuous mode we run a moving average over the predictions to prevent false positives. E.g. if we do 3 classifications per second you’ll see your keyword potentially classified three times (once at the start of the audio file, once in the middle, once at the end). However, if your dataset is unbalanced (there’s a lot more noise / unknown than in your dataset) the ML model typically manages to only find your keyword in the 'center' window, and thus we filter it out as a false positive.

You can fix this by either:

Add more data :-)

Or, disable the moving average filter by going into ei_run_classifier.h (in the edge-impulse-sdk directory) and removing:

    for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
        result->classification[ix].value =
            run_moving_average_filter(&classifier_maf[ix], result->classification[ix].value);
    }

Use the power of the crowd to collect keywords

You can use the power of the crowd to collect your keywords. This is very useful in a STEM or conference setting (here's a video of it in action). For this you'll need an API key, which you'll find under Dashboard > Keys > Add new API key.

Then construct the following URL:

https://smartphone.edgeimpulse.com/keyword.html?apiKey=ei_XXX&sampleLength=30000&keyword=microbit&frequency=11000

Where you replace:

ei_XXX with your API key.
30000 with the desired length that people need to record for in milliseconds (here 30000 = 30 seconds).
microbit with the keyword people should say.
frequency the audio frequency. Keep this at 11KHz for the micro:bit v2.

You can share this link f.e. via a QR code (you generate one here to easily include in a presentation).

NOTE: People will have access to your full project through this API key. Revoke the API key after your presentation!!

jibeer/voice-activated-microbit