Error when using Decimal128
nsalhaji opened this issue · 12 comments
Hi guys,
Curently, I work with Decimal128 supported bu mongodb since 3.4 version.
I use the last version of Jongo and mongo-drive 3.5 version.
Unfortunatly I have this error:
at org.jongo.marshall.jackson.JacksonEngine.unmarshall(JacksonEngine.java:50)
at org.jongo.ResultHandlerFactory$UnmarshallingResultHandler.map(ResultHandlerFactory.java:43)
at org.jongo.MongoCursor.next(MongoCursor.java:46)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:147)
at
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Unknown element type 19
at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@3468ee6e; pos: 137] at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:388)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:348)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1599)
With this content:
{ "_id" : { "$oid" : "5a34277664974b861aaf0423"} , "_class" : "com.allparts.ecommerce.catalog.data.entities.PartInfosEntity" , "price" : { "localizedPrices" : { "FR" : { "value" : { "$numberDecimal" : "30"} , "currency" : "EUR"}}} , "partType" : "IAM" , "vehicleIds" : [ ] , "compatibleIams" : [ ] , "compatibleOes" : [ ] , "partId" : 31 , "creationDateTime" : { "$date" : "2017-12-15T19:50:14.915Z"} , "createdBy" : "Unknown"}
Thanks for your help.
Nizar.
Hi guys,
I have some news, there is fork of bson4jackson supports Decimal128.
So my question is, is there any easy way to integrate this fork to Jongo ?
Or how can we use Jongo with this fork instead of the original library ?
Thanks.
Hello,
This fork renames all packages from de.undercouch.bson4jackson to org.litote.bson4jackson so even if it is compatible with the original version i'm not sure it is possible to use it.
I'm going to look if it is possible to use this fork by using a custom Jongo configuration.
public AbstractMappingBuilder() {
//Theses two first lines tell Jongo to use bson4jackson
this(new ObjectMapper(MongoBsonFactory.createFactory()));
registerModule(new BsonModule());
addModifier(new PropertyModifier());
addModifier(new AnnotationModifier());
}Hi @bguerout,
Thanks man.
I did something quick just to check if Jongo still work with this fork. So I cloned Jongo repo and I replaced all "de.undercouch.bson4jackson" by "org.litote.bson4jackson" and used 3.5 version of embedded mongodb.
Unfortunately there are 44 failed tests with the same Exception :-(
org.bson.BsonSerializationException: While decoding a BSON document 4 bytes were required, but only 0 remain
at org.bson.io.ByteBufferBsonInput.ensureAvailable(ByteBufferBsonInput.java:218)
at org.bson.io.ByteBufferBsonInput.readInt32(ByteBufferBsonInput.java:106)
at org.bson.BsonBinaryWriter.pipe(BsonBinaryWriter.java:303)
at com.mongodb.DBEncoderAdapter.encode(DBEncoderAdapter.java:47)
at com.mongodb.DBEncoderFactoryAdapter.encode(DBEncoderFactoryAdapter.java:33)
at com.mongodb.DBEncoderFactoryAdapter.encode(DBEncoderFactoryAdapter.java:23)
at com.mongodb.CompoundDBObjectCodec.encode(CompoundDBObjectCodec.java:48)
at com.mongodb.CompoundDBObjectCodec.encode(CompoundDBObjectCodec.java:27)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29)
at com.mongodb.connection.UpdateCommandMessage.writeTheWrites(UpdateCommandMessage.java:76)
at com.mongodb.connection.UpdateCommandMessage.writeTheWrites(UpdateCommandMessage.java:43)
at com.mongodb.connection.BaseWriteCommandMessage.encodeMessageBodyWithMetadata(BaseWriteCommandMessage.java:120)
at com.mongodb.connection.RequestMessage.encodeWithMetadata(RequestMessage.java:147)
at com.mongodb.connection.WriteCommandProtocol.sendMessage(WriteCommandProtocol.java:212)
at com.mongodb.connection.WriteCommandProtocol.execute(WriteCommandProtocol.java:93)
at com.mongodb.connection.UpdateCommandProtocol.execute(UpdateCommandProtocol.java:55)
at com.mongodb.connection.UpdateCommandProtocol.execute(UpdateCommandProtocol.java:37)
at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159)
at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:289)
at com.mongodb.connection.DefaultServerConnection.updateCommand(DefaultServerConnection.java:143)
at com.mongodb.operation.UpdateOperation.executeCommandProtocol(UpdateOperation.java:91)
at com.mongodb.operation.BaseWriteOperation$1.call(BaseWriteOperation.java:139)
at com.mongodb.operation.BaseWriteOperation$1.call(BaseWriteOperation.java:133)
at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:424)
at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:415)
at com.mongodb.operation.BaseWriteOperation.execute(BaseWriteOperation.java:133)
at com.mongodb.operation.BaseWriteOperation.execute(BaseWriteOperation.java:60)
at com.mongodb.Mongo.execute(Mongo.java:819)
at com.mongodb.Mongo$2.execute(Mongo.java:802)
at com.mongodb.DBCollection.executeWriteOperation(DBCollection.java:340)
at com.mongodb.DBCollection.replaceOrInsert(DBCollection.java:418)
at com.mongodb.DBCollection.save(DBCollection.java:407)
at org.jongo.Insert.save(Insert.java:49)
at org.jongo.MongoCollection.save(MongoCollection.java:134)
at org.jongo.FindTest.canFindWithStringAsObjectId(FindTest.java:148)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:24)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Hi guys,
Me again. I tried something new.
Starting from 2.7.0 version of bson4jackson I've just cherry-pick this commit
and renamed it as 2.7.0-fork and I changed the dependency version (of bson4jackson) used in Jongo (1.4.0-SNAPSHOT) to use the new one.
Now, all tests passe both for bson4jackson (2.7.0-fork) and Jongo (1.4.0-SNAPSHOT).
After that, in my application, I can use Decimal128, no more Unknown element type 19 Exception. Of course, Decimal128 it's a native Object from bson types, so we need to define JsonDeserializer from Mongo Decimal128 to Java BigDecimal (convertion from BigDecimal to Decimal128 is already managed by the fix).
In Summary, what's your plan about support Decimal128 ?
bson4jackson is no more longer maintained (the last update was about 2 year ago) and the author said "I'm currently on a longer vacation" :-).
Thanks.
Hello,
It should be great if Decimal128 was added to bson4jackson.
Do you think you can create a pull request ?
Hi @nsalhaji ,
I confirm than you can use a custom version of bson4jackson (eg. 2.7.0-fork). This can be a viable solution until a new release is published.
A fork can be tests against all jongo's tests with the following code:
@RunWith(CompatibilitySuite.class)
public class BsonForkCompatibilitySuiteTest {
@Parameters
public static TestContext context() {
Mapper mapper = new JacksonMapper.Builder(new ObjectMapper(BsonForkFactory.createFactory())) //custom class
.registerModule(new BsonForkModule()) //custom class
.addModifier(new PropertyModifier())
.addModifier(new AnnotationModifier())
.build();
return new TestContext("BsonForkMapper", mapper);
}
}I've started to create an example in https://github.com/bguerout/jongo-extras but Jongo seems not compatible with org.litote:bson4jackson-fork:2.8.0 and more important with
de.undercouch:bson4jackson:2.8.0-SNAPSHOT.
Do you have a test with Jongo which shows that using Decimal128 is not possible ?
I'm going to focus on 2.8.0 compatibility because bson4jackson is still maintained and seems to me the only viable solution to create bson document with jackson.
Hi guys,
The next version 2.9.0 of bson4jackson will be released soon, very soon. Of course this new version include support of Decimal128.
More about that here michel-kraemer/bson4jackson#74 (comment)
FYI bson4jackson@2.8.0 (compatible with jackson@2.8.x) and bson4jackson@2.9.0 (compatible with jackson@2.9.x) have been released in early 2018.
Both versions seem to be currently incompatible with jongo (both jongo@1.3.0 and master branch) : 68 tests are currently failing on master when I try to align on bson4jackson@2.8.0 or bson4jackson@2.9.0.
Most of these failures are related to this While decoding a BSON document 4 bytes were required, but only 0 remain error
I tried to debug but I'm missing jongo knowledge to understand what's going on here :/
Ok so i found the origin of the While decoding a BSON document 4 bytes were required, but only 0 remain error.
The problem seems to come from this PR michel-kraemer/bson4jackson#63.
Overriding flush method in MongoBsonGenerator and rollbacking the changes make every tests pass with bson4jackson@2.8.0 or bson4jackson@2.9.0
package org.jongo.marshall.jackson.bson4jackson;
...
class MongoBsonGenerator extends BsonGenerator {
public MongoBsonGenerator(int jsonFeatures, int bsonFeatures, OutputStream out) {
super(jsonFeatures, bsonFeatures, out);
}
@Override
public void flush() throws IOException {
_out.flush();
}
...
}Nevertheless I need to spend more time to understand the root cause of the problem.
Hello 1.3.1 and 1.4.0 have been released.
1.3.1: Jackson fixAcces(true) and Jackson update to 2.7.9
1.4.0: Jackson and bson4jackson updated to 2.9.x and enhancement of Jongo classes extensibility
You can find more informations here: https://github.com/bguerout/jongo/releases
Cool ! Happy to hear that from you Benoit 👍
Will try to align restx on 1.3.1 as quickly as possible and let you know if everything sounds ok from us :)