linkedin/isolation-forest

Unable to save and load model

prarshah opened this issue · 5 comments

prarshah commented

I am using scala 2.11.8 and spark 2.4.0

Exception in thread "main" java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods$.parse(Lorg/json4s/JsonInput;Z)Lorg/json4s/JsonAST$JValue;
	at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter$$anonfun$8.apply(IsolationForestModelReadWrite.scala:302)
	at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter$$anonfun$8.apply(IsolationForestModelReadWrite.scala:301)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
	at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
	at scala.collection.AbstractTraversable.map(Traversable.scala:104)
	at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter.getMetadataToSave(IsolationForestModelReadWrite.scala:301)
	at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter.saveMetadata(IsolationForestModelReadWrite.scala:280)
	at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter.saveImplHelper(IsolationForestModelReadWrite.scala:253)
	at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelWriter.saveImpl(IsolationForestModelReadWrite.scala:241)
jverbus commented

Are you building the project yourself? If so, are you specifying the sparkVersion in the build command?

./gradlew build -PsparkVersion=2.4.3

You can also download a Spark 2.4.3 jar here:
https://bintray.com/linkedin/maven/isolation-forest/0.3.2#files/com%2Flinkedin%2Fisolation-forest%2Fisolation-forest_2.4.3_2.11%2F0.3.2

prarshah commented

Thank you for the quick reply, No I am using the maven dependency

com.linkedin.isolation-forest
isolation-forest_2.3.0_2.11
0.3.2

When I changed the json4s-jackson's version to 3.2.11, it was able to save the model but was unable to read it.

prarshah commented

This is the updated error:

Exception in thread "main" java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods$.parse$default$3()Z
at org.apache.spark.ml.param.IntParam.jsonDecode(params.scala:410)
at org.apache.spark.ml.param.IntParam.jsonDecode(params.scala:390)
at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelReader$Metadata$$anonfun$setParams$1.apply(IsolationForestModelReadWrite.scala:169)
at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelReader$Metadata$$anonfun$setParams$1.apply(IsolationForestModelReadWrite.scala:167)
at scala.collection.immutable.List.foreach(List.scala:392)
at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelReader$Metadata.setParams(IsolationForestModelReadWrite.scala:167)
at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelReader.load(IsolationForestModelReadWrite.scala:60)
at com.linkedin.relevance.isolationforest.IsolationForestModelReadWrite$IsolationForestModelReader.load(IsolationForestModelReadWrite.scala:38)
at org.apache.spark.ml.util.MLReadable$class.load(ReadWrite.scala:380)
at com.linkedin.relevance.isolationforest.IsolationForestModel$.load(IsolationForestModel.scala:140)
at isolation_forest.OutlierDetectorEstimatorTest$.delayedEndpoint$isolation_forest$OutlierDetectorEstimatorTest$1(OutlierDetectorEstimatorTest.scala:60)
at isolation_forest.OutlierDetectorEstimatorTest$delayedInit$body.apply(OutlierDetectorEstimatorTest.scala:9)

jverbus commented

com.linkedin.isolation-forest
isolation-forest_2.4.3_2.11
0.3.2

  • ❤️1