The Core NLP processors don't understand Easter or Mother's Day and throws an exception
Closed this issue · 8 comments
kwalcock commented
See the test case. The closest that our code gets to it is where it re-throws the exception. This breaks Eidos.
kwalcock commented
It seems like the problem may be that something expects an org/joda/time/LocalDate but instead has java/time/LocalDate.
MihaiSurdeanu commented
Can you please paste in the exception trace here?
Also, does the referenced test pass with CluProcessor?
Thanks!
kwalcock commented
It is happening for holidays that aren't on the same day each year. If the year isn't specified, it's OK, but if it is and so the day can be tracked down, it fails. For example, the generic Good Friday is OK, but Good Friday 2023 is not. Christmas 2017 is OK.
CluProcessor doesn't have a problem with crashing.
java.lang.NoSuchMethodError: de.jollyday.Holiday.getDate()Lorg/joda/time/LocalDate;
at edu.stanford.nlp.time.JollyDayHolidays$JollyHoliday.resolveWithYear(JollyDayHolidays.java:211)
at edu.stanford.nlp.time.JollyDayHolidays$JollyHoliday.resolve(JollyDayHolidays.java:223)
at edu.stanford.nlp.time.JollyDayHolidays$JollyHoliday.intersect(JollyDayHolidays.java:197)
at edu.stanford.nlp.time.SUTime$Time.intersect(SUTime.java:1568)
at edu.stanford.nlp.time.SUTime$TemporalOp$7.apply(SUTime.java:1124)
at edu.stanford.nlp.time.SUTime$TemporalOp.apply(SUTime.java:1355)
at edu.stanford.nlp.time.SUTime$TemporalOp.apply(SUTime.java:1360)
at edu.stanford.nlp.time.GenericTimeExpressionPatterns$5.apply(GenericTimeExpressionPatterns.java:362)
at edu.stanford.nlp.ling.tokensregex.types.Expressions$FunctionCallExpression.evaluate(Expressions.java:929)
at edu.stanford.nlp.ling.tokensregex.SequenceMatchRules$SequenceMatchResultExtractor.apply(SequenceMatchRules.java:758)
at edu.stanford.nlp.ling.tokensregex.SequenceMatchRules$SequenceMatchResultExtractor.apply(SequenceMatchRules.java:736)
at edu.stanford.nlp.ling.tokensregex.SequenceMatchRules$SequencePatternExtractRule.apply(SequenceMatchRules.java:1010)
at edu.stanford.nlp.ling.tokensregex.SequenceMatchRules$SequencePatternExtractRule.apply(SequenceMatchRules.java:964)
at edu.stanford.nlp.ling.tokensregex.SequenceMatchRules$CoreMapFunctionApplier.apply(SequenceMatchRules.java:1178)
at edu.stanford.nlp.ling.tokensregex.SequenceMatchRules$CoreMapFunctionApplier.apply(SequenceMatchRules.java:1156)
at edu.stanford.nlp.ling.tokensregex.MatchedExpression$SingleAnnotationExtractor.annotate(MatchedExpression.java:116)
at edu.stanford.nlp.ling.tokensregex.MatchedExpression$SingleAnnotationExtractor.annotate(MatchedExpression.java:108)
at edu.stanford.nlp.ling.tokensregex.MatchedExpression.extractAnnotation(MatchedExpression.java:236)
at edu.stanford.nlp.ling.tokensregex.MatchedExpression.extractAnnotation(MatchedExpression.java:226)
at edu.stanford.nlp.ling.tokensregex.CoreMapExpressionExtractor.annotateExpressions(CoreMapExpressionExtractor.java:542)
at edu.stanford.nlp.ling.tokensregex.CoreMapExpressionExtractor.applyCompositeRule(CoreMapExpressionExtractor.java:432)
at edu.stanford.nlp.ling.tokensregex.CoreMapExpressionExtractor.extractExpressions(CoreMapExpressionExtractor.java:495)
at edu.stanford.nlp.time.TimeExpressionExtractorImpl.extractTimeExpressions(TimeExpressionExtractorImpl.java:198)
at edu.stanford.nlp.time.TimeExpressionExtractorImpl.extractTimeExpressions(TimeExpressionExtractorImpl.java:184)
at edu.stanford.nlp.time.TimeExpressionExtractorImpl.extractTimeExpressionCoreMaps(TimeExpressionExtractorImpl.java:115)
at edu.stanford.nlp.time.TimeExpressionExtractorImpl.extractTimeExpressionCoreMaps(TimeExpressionExtractorImpl.java:105)
at edu.stanford.nlp.ie.regexp.NumberSequenceClassifier.runSUTime(NumberSequenceClassifier.java:345)
at edu.stanford.nlp.ie.regexp.NumberSequenceClassifier.classifyWithSUTime(NumberSequenceClassifier.java:143)
at edu.stanford.nlp.ie.regexp.NumberSequenceClassifier.classifyWithGlobalInformation(NumberSequenceClassifier.java:106)
at edu.stanford.nlp.ie.NERClassifierCombiner.recognizeNumberSequences(NERClassifierCombiner.java:322)
at edu.stanford.nlp.ie.NERClassifierCombiner.classifyWithGlobalInformation(NERClassifierCombiner.java:277)
at edu.stanford.nlp.ie.AbstractSequenceClassifier.classifySentenceWithGlobalInformation(AbstractSequenceClassifier.java:343)
at edu.stanford.nlp.pipeline.NERCombinerAnnotator.doOneSentence(NERCombinerAnnotator.java:368)
at edu.stanford.nlp.pipeline.SentenceAnnotator.annotate(SentenceAnnotator.java:102)
at edu.stanford.nlp.pipeline.NERCombinerAnnotator.annotate(NERCombinerAnnotator.java:310)
at edu.stanford.nlp.pipeline.AnnotationPipeline.annotate(AnnotationPipeline.java:76)
at edu.stanford.nlp.pipeline.StanfordCoreNLP.annotate(StanfordCoreNLP.java:637)
at org.clulab.processors.shallownlp.ShallowNLPProcessor.$anonfun$recognizeNamedEntities$1(ShallowNLPProcessor.scala:244)
at org.clulab.processors.shallownlp.ShallowNLPProcessor.$anonfun$recognizeNamedEntities$1$adapted(ShallowNLPProcessor.scala:239)
at scala.Option.foreach(Option.scala:407)
at org.clulab.processors.shallownlp.ShallowNLPProcessor.recognizeNamedEntities(ShallowNLPProcessor.scala:239)
at org.clulab.processors.shallownlp.ShallowNLPProcessor.annotate(ShallowNLPProcessor.scala:175)
at org.clulab.processors.Processor.annotate(Processor.scala:128)
at org.clulab.processors.Processor.annotate$(Processor.scala:125)
at org.clulab.processors.shallownlp.ShallowNLPProcessor.annotate(ShallowNLPProcessor.scala:33)
at org.clulab.processors.TestHolidays.test(TestHolidays.scala:11)
at org.clulab.processors.TestHolidays.$anonfun$new$3(TestHolidays.scala:34)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.flatspec.AnyFlatSpecLike$$anon$5.apply(AnyFlatSpecLike.scala:1832)
at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
at org.scalatest.TestSuite.withFixture$(TestSuite.scala:195)
at org.scalatest.flatspec.AnyFlatSpec.withFixture(AnyFlatSpec.scala:1686)
at org.scalatest.flatspec.AnyFlatSpecLike.invokeWithFixture$1(AnyFlatSpecLike.scala:1830)
at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$runTest$1(AnyFlatSpecLike.scala:1842)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.flatspec.AnyFlatSpecLike.runTest(AnyFlatSpecLike.scala:1842)
at org.scalatest.flatspec.AnyFlatSpecLike.runTest$(AnyFlatSpecLike.scala:1824)
at org.scalatest.flatspec.AnyFlatSpec.runTest(AnyFlatSpec.scala:1686)
at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$runTests$1(AnyFlatSpecLike.scala:1900)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:390)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:427)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.flatspec.AnyFlatSpecLike.runTests(AnyFlatSpecLike.scala:1900)
at org.scalatest.flatspec.AnyFlatSpecLike.runTests$(AnyFlatSpecLike.scala:1899)
at org.scalatest.flatspec.AnyFlatSpec.runTests(AnyFlatSpec.scala:1686)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.flatspec.AnyFlatSpec.org$scalatest$flatspec$AnyFlatSpecLike$$super$run(AnyFlatSpec.scala:1686)
at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$run$1(AnyFlatSpecLike.scala:1945)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.flatspec.AnyFlatSpecLike.run(AnyFlatSpecLike.scala:1945)
at org.scalatest.flatspec.AnyFlatSpecLike.run$(AnyFlatSpecLike.scala:1943)
at org.scalatest.flatspec.AnyFlatSpec.run(AnyFlatSpec.scala:1686)
at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:47)
at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1321)
at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1315)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1315)
at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:992)
at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:970)
at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1481)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:970)
at org.scalatest.tools.Runner$.run(Runner.scala:798)
at org.scalatest.tools.Runner.run(Runner.scala)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2or3(ScalaTestRunner.java:43)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:26)
MihaiSurdeanu commented
I suspect this is happening because both CoreNLP/SUTime and our date normalization code use JollyDay, but probably different versions... I see two potential solutions:
- Make sure the same version is used in both places.
- Or, disable our date normalization for CoreNLP processors.
What do you think?
kwalcock commented
This is in addition to Thanksgiving, which will crash the SEC project.
kwalcock commented
For processors there's a HolidayNormalizer which uses jollyday. In build.sbt we specify version 0.5.10. It turns out that it overrides the value of 0.4.9 that stanford-nlp wants, as you suspected. There was probably an eviction notice at some time, but in corenlp/build.sbt I finally configured it away after seeing it too often. I'm reverting to 0.4.9 and fixing up the code to use it.
Will this mess up any other project that has processors as a dependency but needs a higher version of jollyday?
MihaiSurdeanu commented
I think this should be fine.