altoo-ag/akka-kryo-serialization

Incremental Id's

Closed this issue · 2 comments

dispalt commented

I noticed incremental id's don't really work with a pool size > 1. You basically run into errors that look like this:

I have serialize-creators and serialize-messages on to show the problem.

If you look below you can see that 1466714 was registered with what I am guessing is a different Kryo instance.

Have you worked around this, seems like a serious limitation to incremental ids.

platform-xweb Implicitly registered class with id: akka.cluster.InternalClusterAction$PublishEvent=1466714
platform-xweb Implicitly registered class with id: akka.cluster.ClusterEvent$ClusterMetricsChanged=826948202
platform-xweb Implicitly registered class with id: scala.collection.immutable.Set$Set1=896475923
platform-xweb Implicitly registered class with id: akka.cluster.NodeMetrics=1069584363
platform-xweb Implicitly registered class with id: akka.actor.Address=1346970375
platform-xweb Implicitly registered class with id: scala.Some=1288426183
platform-xweb Implicitly registered class with id: scala.collection.immutable.HashSet$HashTrieSet=1487055747
platform-xweb Implicitly registered class with id: akka.cluster.Metric=1719173245
platform-xweb Implicitly registered class with id: akka.cluster.EWMA=1296128422
platform-xweb Implicitly registered class with id: scala.None$=1482670802
platform-xweb 23:51:23.121 [ERROR] a.c.ClusterDomainEventPublisher akka://imgress/system/cluster/core/publisher - swallowing exception during message send
platform-xweb com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 1466714
platform-xweb   at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:137) ~[kryo-3.0.3.jar:na]
platform-xweb   at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670) ~[kryo-3.0.3.jar:na]
platform-xweb   at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:781) ~[kryo-3.0.3.jar:na]
platform-xweb   at com.romix.akka.serialization.kryo.KryoBasedSerializer.fromBinary(KryoSerializer.scala:508) ~[akka-kryo-serialization_2.11-0.4.0.jar:0.4.0]
platform-xweb   at com.romix.akka.serialization.kryo.KryoSerializer.fromBinary(KryoSerializer.scala:338) ~[akka-kryo-serialization_2.11-0.4.0.jar:0.4.0]
platform-xweb   at akka.serialization.Serialization$$anonfun$deserialize$3.apply(Serialization.scala:142) ~[akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at scala.util.Try$.apply(Try.scala:192) ~[scala-library-2.11.7.jar:na]
platform-xweb   at akka.serialization.Serialization.deserialize(Serialization.scala:142) ~[akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.actor.dungeon.Dispatch$class.sendMessage(Dispatch.scala:128) ~[akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.actor.ActorCell.sendMessage(ActorCell.scala:374) [akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.actor.Cell$class.sendMessage(ActorCell.scala:295) ~[akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.actor.ActorCell.sendMessage(ActorCell.scala:374) [akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.actor.LocalActorRef.$bang(ActorRef.scala:402) ~[akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.cluster.ClusterMetricsCollector.publish(ClusterMetricsCollector.scala:176) ~[akka-cluster_2.11-2.4.1.jar:na]
platform-xweb   at akka.cluster.ClusterMetricsCollector.collect(ClusterMetricsCollector.scala:139) ~[akka-cluster_2.11-2.4.1.jar:na]
platform-xweb   at akka.cluster.ClusterMetricsCollector$$anonfun$receive$1.applyOrElse(ClusterMetricsCollector.scala:89) ~[akka-cluster_2.11-2.4.1.jar:na]
platform-xweb   at akka.actor.Actor$class.aroundReceive(Actor.scala:480) ~[akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.cluster.ClusterMetricsCollector.aroundReceive(ClusterMetricsCollector.scala:44) ~[akka-cluster_2.11-2.4.1.jar:na]
platform-xweb   at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526) [akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.actor.ActorCell.invoke(ActorCell.scala:495) [akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.dispatch.Mailbox.run(Mailbox.scala:224) [akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [akka-actor_2.11-2.4.1.jar:na]
platform-xweb   at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [scala-library-2.11.7.jar:na]
platform-xweb   at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [scala-library-2.11.7.jar:na]
platform-xweb   at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [scala-library-2.11.7.jar:na]
platform-xweb   at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [scala-library-2.11.7.jar:na]
luben commented

Hi,
I would strongly advise against using incremental ids - they depend on the order of execution/registration. I don't know of any effort to make this better.

dispalt commented

yeah just trying to figure out why you'd even have it, but then I looked at the code and it appears to be using murmur hash on the FQCN.

https://github.com/romix/akka-kryo-serialization/blob/78f96bb7b17b1ceed6cea62dfa7c24ad1b673b20/src/main/scala/com/romix/scala/serialization/kryo/KryoClassResolver.scala#L43

Im still curious about the original question.