Realtime streaming example?
milsun opened this issue · 3 comments
Hey, can you provide an example, as to how we can run stream example as provided in whisper.cpp, with your code in python. Thanks!!!
I haven't really explored the streaming functionality. However, from a brief look achieving this doesn't seem very straight forward. You'd need to look at whisper.cpp/stream/stream.cpp and attempt to reimplement in Cython some of the functionality which interfaces with SDL for the audio capture (could be attempted in pure Python, but it would be easier to deal with the low level data structures). Alternativley you could keep most of stream.cpp and make and expose a new single extern (i.e. transcribe_stream()) a black box, which cython can pick up on and expose to Python, where everything else is handled in c++. But you'd need to figure out a nice way to return the text back to Python.
I'm not quite sure how the example works, unfortunately I can't delve particularly deep right now, but a particular issue will be having the data sent back continuously.
A simple yet inelegant method is simply launching a subprocess instead comprising a modified stream.cpp and capturing stdout in Python.
After having to work with stream.cpp, I see the microphone data is retrieved from SDL iteratively, in parts which are then transcribed by whisper.cpp.
The current implementation will need a way to grab so many seconds of microphone audio data. So it can be transcribed, then all the microphone data needs to be grabbed since the last time, before transcribing again etc.
I would like to get update for this topic