pemistahl/lingua

Should `withPreloadedLanguageModels()` only load specified languages?

Marcono1234 opened this issue · 3 comments

Currently withPreloadedLanguageModels() loads the models for all languages. However, (if I see that correctly), when a LanguageDetector is created only for a specific subset of all languages, then the models of the other languages would never be accessed.

Would it make sense to therefore have withPreloadedLanguageModels() only load the specified languages instead of all?

LanguageDetector, line 388:

private fun preloadLanguageModels() {
    runBlocking {
        languageModels.map { models ->
            async(Dispatchers.IO) { languages.forEach { language -> models[language]?.value } }
         }
    }
}

languages is the selected subset of all languages, and only this subset is preloaded. So what you want is already there. :)

Oh sorry you are completely right! I tried refactoring the code and accidentially changed it to load all languages, and then thought the main branch behaves the same way ...

And sorry for all the issues / pull requests. I am not trying to spam this project. Recently I started playing around with the project to see if and where performance could be improved. I don't have changes in a polished state to submit a pull request, but if you want I could open an issue highlighting areas where performance could be improved. Would you be interested in that?

Yes, of course. I'm always interested in performance improvements, so please go ahead. :)