More advanced LED patterns
markusressel opened this issue · 4 comments
It would be really awesome if ESP32-Rhasspy-Satellite would support custom wake/sleep/error led patterns/animations, like Google or Alexa, possibly even with direction of arrival support lateron.
When playing around on a Matrix Voice I noticed that the LED color change is only in effect after the "wake" audio file has played. I tried to change this, since it seems unintuitive that, even though the wakeword has already been detected and the acknowledgment sound is already playing, the LEDs to not reflect this. Looking at the code this is due to the fact that the HotwordDetectedEvent is triggered after the sound file has been played. Is this part of the original hermes protocol? Or is this something that might be improved somehow?
The next step would be to allow LED animations, and then allow the acknowledgment sound and the LED animation to play in parallel, so they do not block each other.
Any hints whether this is achievable would be very much appreciated
Looking at the code this is due to the fact that the
HotwordDetectedEventis triggered after the sound file has been played. Is this part of the original hermes protocol? Or is this something that might be improved somehow?
The actual issue is that the message for playing audio arrives probably before the hotword/toggle message.
It would not really be an issue, but the Matrix Voice cannot play audio and update leds at the same time.
That is because it uses a "wishbonebus" which reads and writes. When doing this at the same time, it crashes.
My solution was to exlude this by using a semaphore, but it might be possible reading and writing at the same time.
Animations are possible, but as I said there is an issue with the wishbone bus regarding parallel read/write which I was not yet able to solve, if it is even possible to solve.
I have started working on making it possible to play an animation instead of setting a simple color, and this already works fine.
However, I am not very experienced with programming for microcontrollers and I am having a difficult time finding a proper way to be able to process things async at all. I am not sure whether the ESP32 supports real threading at all, and even if it does, using something like coroutines / asyncio could be the more robust approach, given that the wishbonebus is synchronous anyway. Could you give me some links/hints on what you have used when you tried to do this?
I encountered the problem because I was trying to read and write at the same time.
My solution was to avoid this and implement a semaphore, so I don't have links of what I did sadly.
Me nog being able to solve the parallel read/write might also be my limitation and not a whisbone limitation ;)
WIP here: