NeonGeckoCom/NeonCore

Consider per-language module loading

Closed this issue · 5 comments

NeonDaniel commented

Objective

Handle loading STT/TTS modules dynamically per-language

Implementation Goals

  • Expose methods for STT/TTS/Translation engines to advertise their supported languages
  • Handle optionally loading plugins per-language
  • Handle STT/TTS/Translation requests with the appropriate plugin

Open Issues

  • Do we handle language as en, en-US, either?
  • How many plugins can we efficiently load simultaneously?
NeonDaniel commented

A proposed method for plugins reporting supported languages:

In each relevant plugin's base class, define a new property; if the list is empty, default behavior should assume that any requested language is supported for backwards-compat (probably logging a warning). If the list isn't empty, language should be validated at runtime and raise a custom exception (UnsupportedLanguage(ValueError)?) for the caller to handle.

@property
def supported_languages(self) -> list:
    return []
NeonDaniel commented

in chatterbox before OPM we did this for TTS https://github.com/HelloChatterbox/text2speech/blob/master/text2speech/modules/__init__.py#L110

Makes sense for TTS to also have a method to get voice names since many engines do support multiple voices per language. I'm not sure that all plugins have or use voice names though? I guess it would just equate to an empty list for a supported language, should TTS just implement describe_voices that returns a dict and STT a similar method returning a list?

Opened an issue for this in mycroft-core. MycroftAI/mycroft-core#3059

NeonDaniel commented

Not highly relevant to Neon anymore; current strategy is to train Nemo STT/Coqui TTS models as needed