Satellite gets stuck randomly
TimP4w opened this issue · 2 comments
TimP4w commented
I'm running the satellite on a Raspberry Pi 4b (4Gb) with a Respeaker 4mic hat.
Initially I had some issues but I traced them back to a slow SD card that I was using, now everything works fine except when I trigger the wakeword and do not say anything afterwards.
What happens is that after the timeout time, I get a response from my assistant (i'm using the Extended OpenAI Conversation for this test) and then the satellite gets stuck and doesn't process any new wakeword trigger.
To further debug this I've also ran the event server for the 4mic hat and I can see that the LEDs are stuck to "yellow"
Given the code I'm assuming it's either stuck on "StreamingStarted" or "VoiceStarted".
if StreamingStarted.is_type(event.type):
self.color(_YELLOW)
...
elif VoiceStarted.is_type(event.type):
self.color(_YELLOW)I'm pretty sure the issue is neither the event server (LEDs) nor the openwakeword server, since restarting either of them doesn't do anything.
There is no info in the logs
(.venv) timp4w@rpi4:~/wyoming-satellite $ script/run --name 'my satellite' --uri 'tcp://0.0.0.0:10700' --mic-command 'arecord -D plughw:CARD=seeed4micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw' --snd-command 'aplay -D plughw:CARD=Headphones,DEV=0 -r 22050 -c 1 -f S16_LE -t raw' --wake-uri tcp://localhost:10400 --event-uri tcp://localhost:10500
INFO:root:Ready
INFO:root:Connected to services
Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
INFO:root:Connected to server
INFO:root:Waiting for wake word
<- "Hey, Jarvis" here
INFO:root:Waiting for wake word
Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
<- Stuck here, yellow LEDs
I noticed I can get it unstuck, if on Home Assistant I change any configuration of the satellite (e.g. noise suppression level, mute/unmute). After which the LEDs flashes, the logs go further stating that no text was recognized and it's waiting for wake word.
Also, on HA it switches back to "Assist in progress" as if the wake word was triggered again.
WARNING:root:Event(type='error', data={'text': 'No text recognized', 'code': 'stt-no-text-recognized'}, payload=None)
INFO:root:Waiting for wake word
INFO:root:Waiting for wake word
I tried to search if anyone else was having this issue, but couldn't find anything. Am I the only one or is this a known (or unknown) "bug"?
Other infos:
$ uname -r
6.6.20+rpt-rpi-v8
TimP4w commented
Update: apparently it gets stuck also when I actually give a command. It's a bit random. Sometimes after a couple of answers it gets stuck, sometimes it survives more, but eventually it freezes.
E.g.
Logs when it does get stuck
event-handler | DEBUG:root:Event(type='pong', data={'text': None}, payload=None)
wyoming-satellite | DEBUG:root:Detection(name='hey_jarvis_v0.1', timestamp=117324648111)
wyoming-satellite | DEBUG:root:Streaming audio
wyoming-satellite | DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
event-handler | DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
event-handler | DEBUG:root:Event(type='detection', data={'name': 'hey_jarvis_v0.1', 'timestamp': 117324648111}, payload=None)
event-handler | DEBUG:root:Event(type='streaming-started', data=None, payload=None)
event-handler | DEBUG:root:Event(type='transcribe', data={'language': 'en'}, payload=None)
event-handler | DEBUG:root:Event(type='ping', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='pong', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='voice-started', data={'timestamp': 1420}, payload=None)
event-handler | DEBUG:root:Event(type='ping', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='pong', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='voice-stopped', data={'timestamp': 2010}, payload=None)
wyoming-satellite | DEBUG:root:Event(type='transcript', data={'text': '<redacted>'}, payload=None)
wyoming-satellite | INFO:root:Waiting for wake word
wyoming-satellite | DEBUG:root:Detection(name='hey_jarvis_v0.1', timestamp=123327278984)
wyoming-satellite | DEBUG:root:Streaming audio
wyoming-satellite | DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
event-handler | DEBUG:root:Event(type='ping', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='pong', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='transcript', data={'text': '<redacted>'}, payload=None)
event-handler | DEBUG:root:Event(type='streaming-stopped', data=None, payload=None)
event-handler | DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
event-handler | DEBUG:root:Event(type='detection', data={'name': 'hey_jarvis_v0.1', 'timestamp': 123327278984}, payload=None)
wyoming-satellite | DEBUG:root:Event(type='synthesize', data={'text': <redacted>', 'voice': {'name': 'en_US-ryan-medium'}}, payload=None)
wyoming-satellite | DEBUG:root:Connected to snd service
wyoming-snd-external | DEBUG:root:Client connected: 125000090812
wyoming-snd-external | DEBUG:root:Running ['aplay', '-D', 'plughw:CARD=Headphones', '-r', '22050', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
wyoming-snd-external | Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
event-handler | DEBUG:root:Event(type='streaming-started', data=None, payload=None)
event-handler | DEBUG:root:Event(type='transcribe', data={'language': 'en'}, payload=None)
event-handler | DEBUG:root:Event(type='synthesize', data={'text': '<redacted>', 'voice': {'name': 'en_US-ryan-medium'}}, payload=None)
event-handler | DEBUG:root:Event(type='audio-start', data={'rate': 16000, 'width': 2, 'channels': 1, 'timestamp': 0}, payload=None)
event-handler | DEBUG:root:Event(type='audio-stop', data={'timestamp': 1.7093125000000011}, payload=None)
event-handler | DEBUG:root:Event(type='played', data=None, payload=None)
event-handler | DEBUG:root:Event(type='ping', data={'text': None}, payload=None)
Logs when it doesn't get stuck
event-handler | DEBUG:root:Event(type='pong', data={'text': None}, payload=None)
wyoming-satellite | DEBUG:root:Detection(name='hey_jarvis_v0.1', timestamp=98072617220)
wyoming-satellite | DEBUG:root:Streaming audio
wyoming-satellite | DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
event-handler | DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
event-handler | DEBUG:root:Event(type='detection', data={'name': 'hey_jarvis_v0.1', 'timestamp': 98072617220}, payload=None)
event-handler | DEBUG:root:Event(type='streaming-started', data=None, payload=None)
event-handler | DEBUG:root:Event(type='transcribe', data={'language': 'en'}, payload=None)
event-handler | DEBUG:root:Event(type='ping', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='pong', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='voice-started', data={'timestamp': 865}, payload=None)
event-handler | DEBUG:root:Event(type='ping', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='pong', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='voice-stopped', data={'timestamp': 1395}, payload=None)
wyoming-satellite | DEBUG:root:Event(type='transcript', data={'text': ' What time is it?'}, payload=None)
wyoming-satellite | INFO:root:Waiting for wake word
wyoming-satellite | DEBUG:root:Wake word detection occurred during refractory period
event-handler | DEBUG:root:Event(type='transcript', data={'text': ' <redacted>'}, payload=None)
wyoming-satellite | DEBUG:root:Event(type='synthesize', data={'text': '<redacted>', 'voice': {'name': 'en_US-ryan-medium'}}, payload=None)
event-handler | DEBUG:root:Event(type='streaming-stopped', data=None, payload=None)
event-handler | DEBUG:root:Event(type='ping', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='pong', data={'text': None}, payload=None)
event-handler | DEBUG:root:Event(type='synthesize', data={'text': '<redacted>', 'voice': {'name': 'en_US-ryan-medium'}}, payload=None)
event-handler | DEBUG:root:Event(type='audio-start', data={'rate': 16000, 'width': 2, 'channels': 1, 'timestamp': 0}, payload=None)
wyoming-snd-external | DEBUG:root:Client connected: 104411740718
wyoming-satellite | DEBUG:root:Connected to snd service
wyoming-snd-external | DEBUG:root:Running ['aplay', '-D', 'plughw:CARD=Headphones', '-r', '22050', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
wyoming-snd-external | Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
event-handler | DEBUG:root:Event(type='audio-stop', data={'timestamp': 2.1156875000000013}, payload=None)
event-handler | DEBUG:root:Event(type='played', data=None, payload=None)
TimP4w commented
Nevermind... Just figured out that I was missing the --wake-word-name 'hey_jarvis' argument. Probably should have paid a closer attention to the docs.