DataFormatException: incorrect header check
ekfliu opened this issue · 4 comments
if run with akka.actor.serialize-messages = on and post-serialization-transformations = "deflate" for single node serialization test results in error
java.util.zip.DataFormatException: incorrect header check at java.util.zip.Inflater.inflateBytes(Native Method) ~[na:1.8.0_112] at java.util.zip.Inflater.inflate(Inflater.java:259) ~[na:1.8.0_112] at java.util.zip.Inflater.inflate(Inflater.java:280) ~[na:1.8.0_112] at com.romix.akka.serialization.kryo.ZipKryoCompressor.fromBinary(KryoSerializer.scala:131) ~[akka-kryo-serialization_2.11-0.5.0.jar!/:na] at com.romix.akka.serialization.kryo.KryoTransformer$$anonfun$4$$anonfun$apply$2.apply(KryoSerializer.scala:56) ~[akka-kryo-serialization_2.11-0.5.0.jar!/:na] at com.romix.akka.serialization.kryo.KryoTransformer$$anonfun$4$$anonfun$apply$2.apply(KryoSerializer.scala:56) ~[akka-kryo-serialization_2.11-0.5.0.jar!/:na] at com.romix.akka.serialization.kryo.KryoTransformer.fromBinary(KryoSerializer.scala:63) ~[akka-kryo-serialization_2.11-0.5.0.jar!/:na] at com.romix.akka.serialization.kryo.KryoSerializer.fromBinary(KryoSerializer.scala:351) ~[akka-kryo-serialization_2.11-0.5.0.jar!/:na] at akka.serialization.Serialization$$anonfun$deserialize$3.apply(Serialization.scala:204) ~[akka-actor_2.11-2.5.4.jar!/:na] at scala.util.Try$.apply(Try.scala:192) ~[scala-library-2.11.11.jar!/:na] at akka.serialization.Serialization.deserialize(Serialization.scala:204) ~[akka-actor_2.11-2.5.4.jar!/:na] at akka.actor.dungeon.Dispatch$class.serializeAndDeserializePayload(Dispatch.scala:170) ~[akka-actor_2.11-2.5.4.jar!/:na] at akka.actor.dungeon.Dispatch$class.serializeAndDeserialize(Dispatch.scala:150) ~[akka-actor_2.11-2.5.4.jar!/:na] at akka.actor.dungeon.Dispatch$class.sendMessage(Dispatch.scala:133) ~[akka-actor_2.11-2.5.4.jar!/:na] at akka.actor.ActorCell.sendMessage(ActorCell.scala:370) ~[akka-actor_2.11-2.5.4.jar!/:na] at akka.actor.Cell$class.sendMessage(ActorCell.scala:291) ~[akka-actor_2.11-2.5.4.jar!/:na] at akka.actor.ActorCell.sendMessage(ActorCell.scala:370) ~[akka-actor_2.11-2.5.4.jar!/:na] at akka.actor.LocalActorRef.$bang(ActorRef.scala:400) ~[akka-actor_2.11-2.5.4.jar!/:na] at akka.actor.ActorRef.tell(ActorRef.scala:124) ~[akka-actor_2.11-2.5.4.jar!/:na]
This bug occurs even on non-same machine nodes in cluster, same machine, non-same actorSystem.
Could you provide a simple test which reproduces this issue?
I moved away from this dependency, wrote my own kryo-serialization support for akka.
But i was hit by same issue =)
The problem is that Deflator is not threadsafe, here
you should not use it as field.
You should create new Deflator/Inflator on each serialization cycle.
This is the variant i wrote for my project:
`
private byte[] deflate( byte[] bytes ){
if( !compress ) return bytes;
if( COMPRESSION_THRESHOLD > bytes.length )
{
return ByteString.createBuilder()
.putByte( (byte)0 )
.putBytes( bytes )
.result()
.toArray();
}
Deflater deflater = new Deflater( Deflater.BEST_SPEED );
deflater.setInput( bytes );
deflater.finish();
ByteStringBuilder builder = ByteString.createBuilder()
.putByte( (byte)1 );
byte[] output = new byte[BLOCK_SIZE];
while( !deflater.finished() )
{
int n = deflater.deflate( output );
builder.putBytes( output, 0, n );
}
deflater.end();
ByteString result = builder.result();
log.trace( "Compressed from {} to {} bytes", bytes.length, result.size() );
return result.toArray();
}
`
This is (probably) not optimal, but my servers are running for month 24/7 with no incident.