Some operations cause the editor to freeze
Closed this issue · 10 comments
dan-zeman commented
We (with @abishekjs) have conllueditor installed on a server and access it remotely. We observed that sometimes an action causes the editor to freeze and it has to be restarted on the server. The error seems to be connected to joining of tokens – at least the last occurrence was.
We have the following sentence:
# sent_id = indiccorp14 # text = സൗമ്യയുടെ മാതാപിതാക്കളായ കുഞ്ഞിക്കണ്ണൻ, കമല എന്നിവരും സൗമ്യയുടെ രണ്ടാമത്തെ മകൾ ഐശ്വര്യയും ദുരൂഹസാഹചര്യത്തിൽ മരിച്ചിരുന്നു. # translit = saumyayuṭe mātāpitākkaḷāya kuññikkaṇṇan̕, kamala ennivaruṁ saumyayuṭe raṇṭāmatte makaḷ̕ aiśvaryayuṁ durūhasāhacaryattil̕ mariccirunnu. # text_en = Soumya's parents Kunhikannan and Kamala and Soumya's second daughter Aishwarya died under mysterious circumstances. 1 സൗമ്യയുടെ സൗമ്യയുടെ PROPN _ Animacy=Anim|Case=Gen|Number=Sing 2 nmod:poss _ Translit=saumyayuṭe|LTranslit=saumyayuṭe 2 മാതാപിതാക്കളായ മാതാപിതാക്കളായ ADJ _ _ 3 amod _ Translit=mātāpitākkaḷāya|LTranslit=mātāpitākkaḷāya 3 കുഞ്ഞിക്കണ്ണൻ കുഞ്ഞിക്കണ്ണൻ PROPN _ Animacy=Anim|Case=Nom|Number=Sing 7 nsubj _ SpaceAfter=No|Translit=kuññikkaṇṇan̕|LTranslit=kuññikkaṇṇan̕ 4 , , PUNCT _ _ 5 punct _ Translit=,|LTranslit=, 5 കമല കമല PROPN _ Animacy=Anim|Case=Nom|Number=Sing 3 conj _ Translit=kamala|LTranslit=kamala 6-8 എന്നിവരും _ _ _ _ _ _ _ Translit=ennivaruṁ 6 എന്ന് എന്ന് SCONJ _ _ 3 mark _ Translit=ennivara|LTranslit=ennivara 7 ഇവർ ഇവർ PRON _ Case=Nom|Deixis=Prox|Number=Plural|Person=3|PronType=Prs 15 ccomp _ Translit=ennivara|LTranslit=ennivara 8 ഉം ഉം PART _ _ 6 cc _ Translit=uṁ|LTranslit=uṁ 9 സൗമ്യയുടെ സൗമ്യയുടെ PROPN _ Animacy=Anim|Case=Gen|Number=Sing 10 nmod:poss _ Translit=saumyayuṭe|LTranslit=saumyayuṭe 10 രണ്ടാമത്തെ രണ്ടാമത്തെ NUM _ NumForm=Word|NumType=Ord 11 nummod _ Translit=raṇṭāmatte|LTranslit=raṇṭāmatte 11 മകൾ മകൾ NOUN _ Animacy=Anim|Case=Nom|Number=Sing 12 nmod _ Translit=makaḷ̕|LTranslit=makaḷ̕ 12-13 ഐശ്വര്യയും _ _ _ _ _ _ _ Translit=aiśvaryayuṁ 12 ഐശ്വര്യ ഐശ്വര്യ PROPN _ Animacy=Anim|Case=Nom|Number=Sing 15 conj _ Translit=aiśvarya|LTranslit=aiśvarya 13 ഉം ഉം PART _ _ 12 cc _ Translit=uṁ|LTranslit=uṁ 14 ദുരൂഹസാഹചര്യത്തിൽ ദുരൂഹസാഹചര്യം NOUN _ Animacy=Inan|Case=Loc|Number=Sing 15 obl _ Translit=durūhasāhacaryattil̕|LTranslit=durūhasāhacaryaṁ 15 മരിച്ചിരുന്നു മരിക്കുക VERB _ Aspect=Imp|Mood=Ind|Polarity=Pos|Tense=Pres|VerbForm=Fin|Voice=Act 0 root _ SpaceAfter=No|Translit=mariccirunnu|LTranslit=maricciruka 16 . . PUNCT _ _ 15 punct _ SpacesAfter=\n|Translit=.|LTranslit=.
When trying to join two tokens, the editor said "token 11 constitutes a cycle" and froze. The tokens to be merged are 6 and 7, they are now split mistakenly. The log on the server (before I manually killed the editor process) contains this:
2022/12/10 14:33:15 POST EditHandler request from /10.10.24.254:63129
COMMAND [mod join 6] sid: 43
java.lang.NullPointerException
at com.orange.labs.conllparser.ConllSentence.normalise(ConllSentence.java:399)
at com.orange.labs.conllparser.ConllSentence.joinWords(ConllSentence.java:1435)
at com.orange.labs.editor.ConlluEditor.process(ConlluEditor.java:1740)
at com.orange.labs.httpserver.ServeurHTTP$EditHandler.handle(ServeurHTTP.java:239)
at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77)
at jdk.httpserver/sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:82)
at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:80)
at jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:848)
at jdk.httpserver/com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:77)
at jdk.httpserver/sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:817)
at jdk.httpserver/sun.net.httpserver.ServerImpl$DefaultExecutor.execute(ServerImpl.java:201)
at jdk.httpserver/sun.net.httpserver.ServerImpl$Dispatcher.handle(ServerImpl.java:560)
at jdk.httpserver/sun.net.httpserver.ServerImpl$Dispatcher.run(ServerImpl.java:526)
at java.base/java.lang.Thread.run(Thread.java:829)
2022/12/10 14:33:27 POST EditHandler request from /10.10.24.254:63129
COMMAND [mod join 6] sid: 43
2022/12/10 14:34:00 POST EditHandler request from /10.10.24.254:63129
COMMAND [mod delete 6] sid: 43
2022/12/10 14:34:12 POST EditHandler request from /10.10.24.254:63129
COMMAND [mod join 7] sid: 43
jheinecke commented
Thanks for signalling this. I can reproduce it.
Strangely if you reattach token 8 to a new head (7), joining 6 and 7 works without problem. You can use this as a workaround, I'll fix the error these days
jheinecke commented
Apparently an easy one. git pull version 2.19.4 and run mvn install. I'll update the docker image and the release bundle later ASAP.
dan-zeman commented
git pullversion 2.19.4 and runmvn install
Thanks. I did it but it says that some tests failed, hence the build failed. I was not the one who performed the previous installation, so I do not know if they had to solve anything similar, or if this is specific to the new version.
dan-zeman commented
[INFO] Running TestConlluComparator
[ERROR] Tests run: 8, Failures: 6, Errors: 0, Skipped: 0, Time elapsed: 0.058 s <<< FAILURE! - in TestConlluComparator
[ERROR] TestConlluComparator.test01OneGroupForm0 Time elapsed: 0.016 s <<< FAILURE!
org.junit.ComparisonFailure:
CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim-form-0.txt
res: testoutput/sim-form-0.txt
expected:<...tences (Form) 0/4 0[,]0%
> but was:<...tences (Form) 0/4 0[.]0%
>
at TestConlluComparator.test01OneGroupForm0(TestConlluComparator.java:98)
[ERROR] TestConlluComparator.test01OneGroupForm1 Time elapsed: 0.01 s <<< FAILURE!
org.junit.ComparisonFailure:
CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim-form-1.txt
res: testoutput/sim-form-1.txt
expected:<...tences (Form) 0/4 0[,]0%
FORM 1 similar.co...> but was:<...tences (Form) 0/4 0[.]0%
FORM 1 similar.co...>
at TestConlluComparator.test01OneGroupForm1(TestConlluComparator.java:115)
[ERROR] TestConlluComparator.test01OneGroupForm2 Time elapsed: 0.006 s <<< FAILURE!
org.junit.ComparisonFailure:
CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim-form-2.txt
res: testoutput/sim-form-2.txt
expected:<...tences (Form) 0/4 0[,]0%
FORM 1 similar.co...> but was:<...tences (Form) 0/4 0[.]0%
FORM 1 similar.co...>
at TestConlluComparator.test01OneGroupForm2(TestConlluComparator.java:132)
[ERROR] TestConlluComparator.test01OneGroupFormUpos2 Time elapsed: 0.004 s <<< FAILURE!
org.junit.ComparisonFailure:
CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim-form-0-upos-2.txt
res: testoutput/sim-form-0-upos-2.txt
expected:<...tences (Form) 0/4 0[,]0%
FORM 1 similar.co...> but was:<...tences (Form) 0/4 0[.]0%
FORM 1 similar.co...>
at TestConlluComparator.test01OneGroupFormUpos2(TestConlluComparator.java:166)
[ERROR] TestConlluComparator.test02TwoForm0 Time elapsed: 0.002 s <<< FAILURE!
org.junit.ComparisonFailure:
CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim2-form-0.txt
res: testoutput/sim2-form-0.txt
expected:<...ences (Form) 2/4 50[,]0%
FORM 0 Aviator, u...> but was:<...ences (Form) 2/4 50[.]0%
FORM 0 Aviator, u...>
at TestConlluComparator.test02TwoForm0(TestConlluComparator.java:183)
[ERROR] TestConlluComparator.test02TwoForm2 Time elapsed: 0.007 s <<< FAILURE!
org.junit.ComparisonFailure:
CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim2-form-2.txt
res: testoutput/sim2-form-2.txt
expected:<...ences (Form) 2/4 50[,]0%
FORM 0 Aviator, u...> but was:<...ences (Form) 2/4 50[.]0%
FORM 0 Aviator, u...>
at TestConlluComparator.test02TwoForm2(TestConlluComparator.java:198)
[INFO] Running TestConllFile
[INFO] Tests run: 46, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.432 s - in TestConllFile
[INFO] Running TestConllSentence
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 s - in TestConllSentence
[INFO] Running TestConlluPlus
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.119 s - in TestConlluPlus
[DEBUG] Closing the fork 1 after saying GoodBye.
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestConlluComparator.test01OneGroupForm0:98 CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim-form-0.txt
res: testoutput/sim-form-0.txt
expected:<...tences (Form) 0/4 0[,]0%
> but was:<...tences (Form) 0/4 0[.]0%
>
[ERROR] TestConlluComparator.test01OneGroupForm1:115 CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim-form-1.txt
res: testoutput/sim-form-1.txt
expected:<...tences (Form) 0/4 0[,]0%
FORM 1 similar.co...> but was:<...tences (Form) 0/4 0[.]0%
FORM 1 similar.co...>
[ERROR] TestConlluComparator.test01OneGroupForm2:132 CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim-form-2.txt
res: testoutput/sim-form-2.txt
expected:<...tences (Form) 0/4 0[,]0%
FORM 1 similar.co...> but was:<...tences (Form) 0/4 0[.]0%
FORM 1 similar.co...>
[ERROR] TestConlluComparator.test01OneGroupFormUpos2:166 CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim-form-0-upos-2.txt
res: testoutput/sim-form-0-upos-2.txt
expected:<...tences (Form) 0/4 0[,]0%
FORM 1 similar.co...> but was:<...tences (Form) 0/4 0[.]0%
FORM 1 similar.co...>
[ERROR] TestConlluComparator.test02TwoForm0:183 CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim2-form-0.txt
res: testoutput/sim2-form-0.txt
expected:<...ences (Form) 2/4 50[,]0%
FORM 0 Aviator, u...> but was:<...ences (Form) 2/4 50[.]0%
FORM 0 Aviator, u...>
[ERROR] TestConlluComparator.test02TwoForm2:198 CoNLL-U comparison
ref: file:/home/conlluadmin/conllueditor/target/test-classes/sim2-form-2.txt
res: testoutput/sim2-form-2.txt
expected:<...ences (Form) 2/4 50[,]0%
FORM 0 Aviator, u...> but was:<...ences (Form) 2/4 50[.]0%
FORM 0 Aviator, u...>
[INFO]
[ERROR] Tests run: 113, Failures: 6, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 36.247 s
[INFO] Finished at: 2022-12-12T20:55:28Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project ConlluEditor: There are test failures.
dan-zeman commented
I figured out that I can do mvn install -DskipTests and it worked, the editor is running again. But perhaps the failing comparison test deserves to become another issue?
jheinecke commented
Strange, this is a localisation error (0.0 vs 0,0) no idea why this occurs now, since the test has been there for ages. I'll try to find the cause
jheinecke commented
try 2.19.5 :-)
dan-zeman commented
Thanks! We got futher :-) but not to the successful end, yet.
[ERROR] Failures: [ERROR] TestConllFile.test40conditionvalidation:574 validation incorrect ref: file:/home/conlluadmin/conllueditor/target/test-classes/validrules.result.txt res: testoutput/validrules.result.txt expected:<...ud-train-767 Om du k[?per en halv liter mj?lk f?r 0:78, en limpa f?r 2:57 och ett halvt kilo margarin f?r 2:87 g?r detta sammanlagt 6:22. Applying rule 5: Upos:NUM == Deprel:nummod ERROR 9 0:78 0:78 NUM RG|NOM Case=Nom|NumType=Card 3 obl 3:obl:f?r SpaceAfter=No ERROR 14 2:57 2:57 NUM RG|NOM Case=Nom|NumType=Card 12 orphan 10.1:obl:f?r Enhanced=obl ERROR 21 2:87 2:87 NUM RG|NOM Case=Nom|NumType=Card 19 orphan 15.1:obl:f?r Enhanced=obl Sentence gl_ctg-ud-dev.conllu 625 D?selle nova redacci?]n a o punto 2 App...> but was:<...ud-train-767 Om du k[?per en halv liter mj?lk f?r 0:78, en limpa f?r 2:57 och ett halvt kilo margarin f?r 2:87 g?r detta sammanlagt 6:22. Applying rule 5: Upos:NUM == Deprel:nummod ERROR 9 0:78 0:78 NUM RG|NOM Case=Nom|NumType=Card 3 obl 3:obl:f?r SpaceAfter=No ERROR 14 2:57 2:57 NUM RG|NOM Case=Nom|NumType=Card 12 orphan 10.1:obl:f?r Enhanced=obl ERROR 21 2:87 2:87 NUM RG|NOM Case=Nom|NumType=Card 19 orphan 15.1:obl:f?r Enhanced=obl Sentence gl_ctg-ud-dev.conllu 625 D?selle nova redacci?]n a o punto 2 App...> [INFO] [ERROR] Tests run: 113, Failures: 1, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------
jheinecke commented
Looks again like a Locale error. Try to set export LC_ALL=C.UTF-8. In fact the utf8 bit is the crucial thing. I do not yet understand why nobody saw this before
jheinecke commented
in fact there was a missing in this test, I fixed and pushed it. Should be OK for any LC_ALL/LANG value.