KRYO throwing NPE with AKKA

Question

KRYO throwing NPE with AKKA

Closed this issue 5 years ago · 11 comments

Hi,

I am using KRYO serilization in AKKA with kryo version 2.24, akka-kryo-serialization_2.10-0.3.2 and akka 2.3.4
I was sending Message containing HashMap from one node to another and it was throwing the NPE.
After reading the article EsotericSoftware/kryo#216 I came to know that there is problem with kryo version i.e. version 2.24. So I tried with Kryo 2.22 and akka-kryo-serialization_2.10-0.3.0 and it worked fine if I run the program via main method. I was happy

but again when I put the code inside Osgi bundle then it's throwing the below exception inside KARAF

org.osgi.framework.BundleException: Activator start error in bundle PMED_Cluster.TestCollector [127].
at org.apache.felix.framework.Felix.activateBundle(Felix.java:2027)[org.apache.felix.framework-4.0.3.jar:]
at org.apache.felix.framework.Felix.startBundle(Felix.java:1895)[org.apache.felix.framework-4.0.3.jar:]
at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:944)[org.apache.felix.framework-4.0.3.jar:]
at org.apache.felix.fileinstall.internal.DirectoryWatcher.startBundle(DirectoryWatcher.java:1263)[6:org.apache.felix.fileinstall:3.2.8]
at org.apache.felix.fileinstall.internal.DirectoryWatcher.startBundles(DirectoryWatcher.java:1235)[6:org.apache.felix.fileinstall:3.2.8]
at org.apache.felix.fileinstall.internal.DirectoryWatcher.process(DirectoryWatcher.java:524)[6:org.apache.felix.fileinstall:3.2.8]
at org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:308)[6:org.apache.felix.fileinstall:3.2.8]
Caused by: java.lang.NoClassDefFoundError: com/esotericsoftware/shaded/org/objenesis/strategy/InstantiatorStrategy
at java.lang.Class.getDeclaredConstructors0(Native Method)[:1.7.0_51]
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2493)[:1.7.0_51]
at java.lang.Class.getConstructor0(Class.java:2803)[:1.7.0_51]
at java.lang.Class.getDeclaredConstructor(Class.java:2053)[:1.7.0_51]
at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$2.apply(DynamicAccess.scala:76)
at scala.util.Try$.apply(Try.scala:161)
at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:73)
at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84)
at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84)
at scala.util.Success.flatMap(Try.scala:200)
at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:84)
at akka.serialization.Serialization.serializerOf(Serialization.scala:165)
at akka.serialization.Serialization$$anonfun$3.apply(Serialization.scala:174)
at akka.serialization.Serialization$$anonfun$3.apply(Serialization.scala:174)
at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:722)
at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224)
at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403)
at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:721)
at akka.serialization.Serialization.(Serialization.scala:174)
at akka.serialization.SerializationExtension$.createExtension(SerializationExtension.scala:15)
at akka.serialization.SerializationExtension$.createExtension(SerializationExtension.scala:12)
at akka.actor.ActorSystemImpl.registerExtension(ActorSystem.scala:711)
at akka.actor.ExtensionId$class.apply(Extension.scala:79)
at akka.serialization.SerializationExtension$.apply(SerializationExtension.scala:12)
at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:175)
at akka.cluster.ClusterActorRefProvider.init(ClusterActorRefProvider.scala:58)
at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:618)
at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:615)
at akka.actor.ActorSystemImpl._start(ActorSystem.scala:615)
at akka.actor.ActorSystemImpl.start(ActorSystem.scala:632)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:141)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:125)
at akka.osgi.OsgiActorSystemFactory.createActorSystem(OsgiActorSystemFactory.scala:33)
at akka.osgi.OsgiActorSystemFactory.createActorSystem(OsgiActorSystemFactory.scala:25)
at pmedConsumer.startCluster.start(startCluster.java:47)
at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:645)
at org.apache.felix.framework.Felix.activateBundle(Felix.java:1977)
... 6 more
Caused by: java.lang.ClassNotFoundException: com.esotericsoftware.shaded.org.objenesis.strategy.InstantiatorStrategy not found by akka-kryo-serialization_2.10 [54]
at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1460)[org.apache.felix.framework-4.0.3.jar:]
at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72)[org.apache.felix.framework-4.0.3.jar:]
at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)[:1.7.0_51]

Is the NPE problem of HashMap and ArrayList is resolved in 3.0 version?? (I tried but it was giving NPE again)
Can you please let me know why it is not working in KARAF ?
Below is my Application.conf for reference

akka {
event-handlers = ["akka.event.slf4j.Slf4jEventHandler"]
loggers = ["akka.event.slf4j.Slf4jLogger"]
loglevel = "DEBUG"
stdout-loglevel = "DEBUG"
extensions = ["com.romix.akka.serialization.kryo.KryoSerializationExtension$"]
actor {
provider = "akka.cluster.ClusterActorRefProvider"

serializers {
#java = "akka.serialization.JavaSerializer"
kryo = "com.romix.akka.serialization.kryo.KryoSerializer"
#mapSer = "com.esotericsoftware.kryo.serializers.MapSerializer"
}

        serialization-bindings {

        "java.util.HashMap" = kryo
         "com.xyz.message.IMessage$RecordNewMessage" = kryo
         "com.xyz.message.TestObject" = kryo
        }   


kryo  {  
type = "nograph"  
idstrategy = "incremental"  
serializer-pool-size = 20
buffer-size = 92427360  
max-buffer-size = -1
use-manifests = false
compression = off
implicit-registration-logging = true 
kryo-trace = false
#kryo-custom-serializer-init = "CustomKryoSerializerInitFQCN"

 mappings {  
  "java.util.HashMap" = 26
 # "java.util.ArrayList" = 26
 #"com.xyz.message.RecordNewMessage" =25
 "com.xyz.message.IMessage$RecordNewMessage" = 25
 "com.xyz.message.TestObject"= 27
}  
kryo-reference-map = false 

classes = [  
#"com.xyz.message.RecordNewMessage",
#"java.util.HashMap"
    # fully.qualified.classname1
    # fully.qualified.classname2            
    ]

}

}
remote {
log-remote-lifecycle-events = on
netty.tcp {
hostname = "127.0.0.1"
port = 0
maximum-frame-size = 30809120
#52428800
send-buffer-size = 92427360
receive-buffer-size = 92427360
}
}

cluster {
seed-nodes = [
"akka.tcp://ClusterSystem@127.0.0.1:2551",
"akka.tcp://ClusterSystem@127.0.0.1:2552"]

auto-down-unreachable-after = 300s

}

Answer 1 · 2014-11-26T18:17:06.000Z

It looks like your real problem is this part of the trace:
"Caused by: java.lang.ClassNotFoundException: com.esotericsoftware.shaded.org.objenesis.strategy.InstantiatorStrategy"

Looks like you miss some of Kryo dependencies in your deployment. Basically, some version of Kryo shaded objenesis into Kryo jars. Later on, it was decided to avoid doing it by default and hence such dependencies should be included in addition to Kryo jar.

As for EsotericSoftware/kryo#216, I'd like to better understand your real problem with the NPE and solve it properly. HashMaps should not be such a problem for Kryo. If you could provide something that reproduces this NPE, I could try to fix it in a proper way.

Answer 2 · 2014-11-26T18:41:25.000Z

Hi,
HashMap and ArryList are working if I run the program through main() method using older kryo version i.e. 2.22 and akka-kryo-serialization_2.10-0.3.0
apart from that i have added the objenesis-2.1.jar also

I I use kryo version 2.24 and akka-kryo-serialization_2.10-0.3.2 and When one akka node try to send message containing the hashmap to another node then it throws NPE

Dependencies are resolved inside KARAF aswell and com.esotericsoftware.shaded.org.objenesis.strategy.InstantiatorStrategy class is already present inside the kryo2.22.jar but still it's throwing exception

Following jars I have added inside karaf

akka-kryo-serialization_2.10-0.3.0.jar
kryo-2.22.jar
objenesis-2.1.jar

What else Do i need to add in karaf???

I can share the code which can reproduce the scenario.
Sorry but I don't know how I can send or attache my code.

Answer 3 · 2014-11-26T19:13:18.000Z

I can share the code which can reproduce the scenario.
Sorry but I don't know how I can send or attache my code.

Yes, github does not make it easy to attach the code as an archive. So, the options are:

put the code on github as a repo and provide a link to this repo
upload your code somewhere, provide me with a link so that I can download it (and then remove the uploaded archive, if you don't want everyone to access it afterwards)....
on google groups, there is a possibility to send a private message with attachments, IIRC. This way, it is only me who can access your code.

Answer 4 · 2014-11-27T04:26:32.000Z

Hi Romix

I have shared the code in google drive
Below is the link
https://drive.google.com/folderview?id=0B5BbhPQEq7RNbjF0aXdpbjhjR0E&usp=sharing

I hope you can access it

Kryo code.zip contains code that you can run from main class StartClusterTransformer.java
(runs with kryo 2.24 jar, it will throw NPE)

Kryo code.zip :- contains code to run in OSGI (throwing exception with both kryo 2.22 and 2.24)

Answer 5 · 2014-11-27T04:53:09.000Z

Thanks! I've downloaded the code. If you want, you can remove it.

Answer 6 · 2014-11-27T09:32:23.000Z

Hi Romix,

I was getting exception java.lang.ClassNotFoundException: com.esotericsoftware.shaded.org.objenesis.strategy.InstantiatorStrategy not found by akka-kryo-serialization_2.10

and I am using Kryo-2.22.jar. It's like class is present inside the jar but is not visible to akka-kryo-serialization_2.10 v0.3.0 jar

So what I have done is I have manually edited the manifest file of kryo-2.22.jar and exported the above package
Export-Package: com.esotericsoftware.shaded.org.objenesis.strategy;version="2.22.0",

and it's working fine.

Kryo is working fine with akka and osgi.

So is this some kind of bug in kryo or do they need to add this support??

Answer 7 · 2014-11-27T15:10:53.000Z

Hmm. That's interesting. It could be that the manifest file of kryo-2.22.jar was incomplete. You can mention it on the original issue you filed on Kryo.

In the meantime, I'll try to see why kryo version 2.24, akka-kryo-serialization_2.10-0.3.2 and akka 2.3.4 do not work for you.

Answer 8 · 2014-11-27T15:52:44.000Z

I'm trying to reproduce your problem using the archives you provided.

Kryo code.zip seems to contain a very incomplete gradle which would not allow it to be built because it lacks any dependencies. I guess your just archived your Eclipse project.

Kryo code OSGI.zip is more complete. I can build all three projects from there. But how and in which order do I start the whole thing? Is it possible to do it from gradle? Or do I need to do something from Eclipse? Please let me know how to reproduce the problem. I'm interested in reproducing a problem with kryo 2.24.

BTW, why you don't use the latest version of Kryo?

Answer 9 · 2014-11-27T15:54:51.000Z

BTW, why you don't use the latest version of Kryo?

OK. This is probably due to the fact that akka-kryo-serialization depends on kryo-2.24. I see.

Answer 10 · 2014-11-28T03:56:56.000Z

In Kryo code.zip just put the dependencies(akka, kryo, scala, slf4j) in the classpath, and put common project as dependency project in other two.
After putting dependencies just run StartClusterTransformer.java

Kryo code OSGI.zip is complete gradle project, just build it and put in karaf and reslove all dependencies. It will work but only for kryo-2.22.jar with akka-kryo-serialization_2.10-0.3.0.jar

In both projects you can reproduce the exception by using kryo 2.24 and akka-kryo-serialization_2.10-0.3.2.jar

Answer 11 · 2014-12-02T04:29:01.000Z

saviour23, there are some bugfixes in the current unreleased yet akka-kryo-serialization that may be addressing your problem. What happens if you try to compile it from the current master? It will depend on kryo-2.24.0 and objenesis-2.1.

If you can't build it from sources, another idea is to try to manually register all the types use use like:

  kryo {
      type = "nograph"
      idstrategy = "incremental"
      buffer-size = 65536
      serializer-pool-size = 16
      kryo-reference-map = false
      implicit-registration-logging = true
      compression = lz4
      mappings {    # non-default type mappings
        "scala.collection.immutable.TreeMap"                = 20
        "scala.collection.immutable.HashMap$HashTrieMap"    = 21
        "scala.collection.immutable.HashMap$HashMap1"       = 22
        "scala.collection.immutable.Vector"                 = 23
     }
     ...