morfologik/morfologik-stemming

InvalidMarkExceptions thrown from BufferUtils

Closed this issue · 4 comments

nbelisle11 commented

I tried to dig into this one but I can't figure out why this is happening based on source code analysis nor can I reliably reproduce it. It seems to be happening intermittently. Error & full stacktrace:
java.lang.RuntimeException: Could not check sentence (language: English (US)): 'mixture'

java.nio.InvalidMarkException: null
    at java.nio.Buffer.reset(Buffer.java:306)
    at morfologik.stemming.BufferUtils.charsToBytes(BufferUtils.java:177)
    at morfologik.speller.Speller.charSequenceToBytes(Speller.java:195)
    at morfologik.speller.Speller.isInDictionary(Speller.java:259)
    at morfologik.speller.Speller.isMisspelled(Speller.java:239)
    at org.languagetool.rules.spelling.morfologik.MorfologikSpeller.isMisspelled(MorfologikSpeller.java:86)
    at org.languagetool.rules.spelling.morfologik.MorfologikMultiSpeller.isMisspelled(MorfologikMultiSpeller.java:129)
    at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.isMisspelled(MorfologikSpellerRule.java:157)
    at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.getRuleMatches(MorfologikSpellerRule.java:178)
    at org.languagetool.rules.en.AbstractEnglishSpellerRule.getRuleMatches(AbstractEnglishSpellerRule.java:45)
    at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.match(MorfologikSpellerRule.java:107)
    at org.languagetool.JLanguageTool.checkAnalyzedSentence(JLanguageTool.java:644)
    at org.languagetool.JLanguageTool$TextCheckCallable.call(JLanguageTool.java:991)
    at org.languagetool.JLanguageTool$TextCheckCallable.call(JLanguageTool.java:943)
    at org.languagetool.JLanguageTool.performCheck(JLanguageTool.java:606)
    at org.languagetool.JLanguageTool.check(JLanguageTool.java:558)
    at org.languagetool.JLanguageTool.check(JLanguageTool.java:508)
    at org.languagetool.JLanguageTool.check(JLanguageTool.java:504)

Might be related in some way to #75 and/or #90

dweiss commented

Make sure speller is not used in a multi-threaded environment. Looks like you're calling it from multiple threads concurrently and it messes up internal buffers.

dweiss commented

Let me know if this was indeed the case.

nbelisle11 commented

We are indeed using this in a multithreaded environment. I'll adjust and reopen if this doesn't resolve the issue. Thanks for the prompt response!