Streaming - real-time from twilio
Closed this issue · 4 comments
Any plans on adding support to realtime transcriptions? Real-time AssemblyAI python examples are tough to implement in a productive way. I mean for other useful things than just print text. Cause I dwell on performance issues related to multiple websockets inside FastAPI framework, which rescricts a little websockets libraries possibilities if I am not wrong it only allows async websockets, not threaded, at least when considering high performance. Please correct me if I said anything wrong, I am learning as I go.
Basically what I am doing and have thought as being very unperformant (and difficult to optimize) is creating a websocket from twilio to python, from python to assembly AI, and add some printing after each Transcription is not partial but full.
It's taking too long inside fastapi. I am talking about 3-5 seconds at first, 20 seconds in the long run to print final sentence after I stop speaking. I am sure it's my code to blame, but I wouldn't need to go through bad code if it was already bundled good code where we can easily input and output things, realtime, and choose when to block, and when not to.
If this feels like a feature I would love to add it and collaborate on a PR.
I also propose enhancing suchproject with the incorporation of sentence emotion and entity extraction within the transcripts. It would be beneficial to include these additional elements within the partialTranscript and finalTranscript parts. Maybe something like, finalEmotions and finalEntities objects. That gets output faster somehow.
However, I currently don't perceive a strong use case for including partialEmotions or partialEntities. If anyone can see potential benefits, please share your thoughts.
Hi @perinm thank you so much for taking the time to suggest this improvement to the SDK. We were busy adding real-time to the SDK and the good news is this is now available in the 0.11.0 release. There are a couple of examples on the PyPI page and we will soon be updating our documentation to show how to use real-time ASR with SDK examples rather than the more verbose no-dependencies code examples that are currently on there.
Please let us know how this works for you and if you want to talk further you can contact me directly at makai@assemblyai.com.