Sharrnah/whispering

Faster Whisper (CTranslate2) Support

Infinitay opened this issue · 3 comments

Infinitay commented

https://github.com/guillaumekln/faster-whisper/

This implementation is up to 4 times faster than openai/whisper for the same accuracy while using less memory

I would assume we'd greatly benefit from the improved performance since we're working with live translating

Sharrnah commented

thanks. this is now integrated and possible to activate in the UI (or setting faster_whisper: true in the config yaml)

grafik

  • 🚀1
Infinitay commented

Honestly I was not expecting support for this, thank you so much. I'm looking forward to testing and using this soon. Hopefully I can finally run the large whisper model and get better results.

  • ❤️1
Sharrnah commented

thanks. Yes its so much faster in my testings that i could not not implement it. :)

Another setting i noticed that can help a bit more in speeding up translations or transcriptions is disabling temperature_fallback. I added that option in the last update as well.

I moved it to advanced because it can reduce the quality since it only does a single iteration even if the A.I. isn't perfectly confident about the result, and the CTranslate2 version is possibly still fast enough.

grafik

And don't hesitate to ask anything or make another suggestion. Always happy to look into it.