Interruption of assistant in VAD mode not working

Question

Interruption of assistant in VAD mode not working

Opened this issue a year ago · 4 comments

SuperNiceDev commented a year ago

In VAD mode it seems you cannot interrupt the assistant. It makes a short break, but then keeps on going.

Answer 1 · 2024-10-22T12:25:47.000Z

I agree, this seems to be a common issue. It also seems to interrupt itself a lot.

Answer 2 · 2024-10-22T14:02:05.000Z

Thanks for the confirmation.

Answer 3 · 2024-10-22T18:39:38.000Z

I was able to get it working pretty well. Heres how

System constantly processes microphone input at regular intervals (every 0.1 seconds)
Input is converted to PCM16 format and sent to OpenAI's server
Server performs Voice Activity Detection (VAD)

User speaks during AI audio →
→ Server detects speech ("input_audio_buffer.speech_started" event) →
→ AudioManager stops current audio playback
→ System cancels active response
→ All states reset (isResponseActive, activeResponseId, etc.)
→ System ready for new interaction

Answer 4 · 2024-12-03T15:55:33.000Z

are you hosting the client on an external server? Or are you implementing it client side?