In the first (step 0) is just proof of concept. We're experimenting with microphones and trying to detect a ball bouncing on the table. Currently, as seen on the waveforms above, we can quite reliably visually detect solid hits in the waveform (the recordings were made outside, with car noise, music and conversations in the background). Smaller hits, such as the ball quickly bouncing off from the net are trickier, but still well-discernable. The microphone in both instances is firmly attached by the head to the bottom of the table. Using the .ino file we track the output of the microphone via the Arduino ADC, and simply print it bytewise to the serial, from where the serialread.py script reads it and outputs it to a .wav file. Then, via findpeaks.py we can pretty reliably detect individual ball hits. Currently we just print them out, the point here is just to prove we can do it.
Peaks found:
Peaks found:
Throughout the development of the project, we decided to use Edge Impulse as our AI-model of choice and, because of the need for continious classification, to transfer the data to the microcontroller via DMA (direct memory access) and not RTOS (real-time operation system).
At first, we used Arduino Uno to collect the samples required (tablehit -- ball hitting the left part of the tennis table, oppositehit -- ball hitting the right part of the table, silence -- no sound of hitting the table (but any other sound is allowed)). After the collection of the samples and training the model using Edge Impulse service, we used Edge Impulse CLI and its Data Forwarder to test our mechanism of recognizing samples. Before that, we had to properly cut our samples so that the model would recognize them with reasonable frequency, and got the accuracy of almost 90%.
Having completed the successful deployment of our AI-model to the microcontroller itself (as CMSIS-PACK), we started retraining the model on the samples collected using the STM32 ADC (analog-to-digital converter) and got pretty similar results (accuracy ~ 90%).
We used Nokia LCD 5110 as the display, STM32F3DISCO as our microcontoroller of choice and KY038 as microphone to recognize the sound.
The game logic (according to the rules of table tennis) was implemented using the FSM (finite state machine). Here is the diagram of states of the game logic:
Connection schema for the display and the microphone:
