Error encountered in ConfigBL.batchJobBL.upsert

Question

Error encountered in ConfigBL.batchJobBL.upsert

Closed this issue 3 years ago · 10 comments

The following stack trace have been encountered in master node launcher when trying to call ConfigBL.batchJobBL.upsert

Exception in thread "main" org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class it.agilelab.bigdata.wasp.models.KeyValueOption.
at org.bson.internal.CodecCache.getOrThrow(CodecCache.java:47)
at org.bson.internal.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:65)
at org.bson.internal.ChildCodecRegistry.get(ChildCodecRegistry.java:52)
at org.mongodb.scala.bson.codecs.IterableCodec.org$mongodb$scala$bson$codecs$IterableCodec$$writeValue(IterableCodec.scala:71)
at org.mongodb.scala.bson.codecs.IterableCodec$$anonfun$writeIterable$1.apply(IterableCodec.scala:87)
at org.mongodb.scala.bson.codecs.IterableCodec$$anonfun$writeIterable$1.apply(IterableCodec.scala:87)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.mongodb.scala.bson.codecs.IterableCodec.writeIterable(IterableCodec.scala:87)
at org.mongodb.scala.bson.codecs.IterableCodec.org$mongodb$scala$bson$codecs$IterableCodec$$writeValue(IterableCodec.scala:69)
at org.mongodb.scala.bson.codecs.IterableCodec.encode(IterableCodec.scala:58)
at org.mongodb.scala.bson.codecs.IterableCodec.encode(IterableCodec.scala:51)
at org.bson.codecs.EncoderContext.encodeWithChildContext(EncoderContext.java:91)
at org.mongodb.scala.bson.codecs.macrocodecs.MacroCodec$class.writeValue(MacroCodec.scala:168)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$$anon$7$KeyValueModelMacroCodec$3.writeValue(VersionedProviders.scala:114)
at org.mongodb.scala.bson.codecs.macrocodecs.MacroCodec$class.writeFieldValue(MacroCodec.scala:159)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$$anon$7$KeyValueModelMacroCodec$3.writeFieldValue(VersionedProviders.scala:114)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$$anon$7$KeyValueModelMacroCodec$3.writeCaseClassData(VersionedProviders.scala:114)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$$anon$7$KeyValueModelMacroCodec$3.writeCaseClassData(VersionedProviders.scala:114)
at org.mongodb.scala.bson.codecs.macrocodecs.MacroCodec$class.writeValue(MacroCodec.scala:165)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$$anon$7$KeyValueModelMacroCodec$3.writeValue(VersionedProviders.scala:114)
at org.mongodb.scala.bson.codecs.macrocodecs.MacroCodec$class.encode(MacroCodec.scala:92)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$$anon$7$KeyValueModelMacroCodec$3.encode(VersionedProviders.scala:114)
at it.agilelab.bigdata.wasp.repository.mongo.providers.AbstractCodecProvider.writeObject(AbstractCodecProvider.scala:77)
at it.agilelab.bigdata.wasp.repository.mongo.providers.DataStoreConfCodecProviders$KeyValueDataStoreConfCodecProvider$.encodeClass(DataStoreConfCodecProviders.scala:84)
at it.agilelab.bigdata.wasp.repository.mongo.providers.DataStoreConfCodecProviders$KeyValueDataStoreConfCodecProvider$.encodeClass(DataStoreConfCodecProviders.scala:65)
at it.agilelab.bigdata.wasp.repository.mongo.providers.AbstractCodecProvider$$anon$1.encode(AbstractCodecProvider.scala:27)
at it.agilelab.bigdata.wasp.repository.mongo.providers.SealedTraitCodecProvider.createBsonDocument(SealedTraitCodecProvider.scala:41)
at it.agilelab.bigdata.wasp.repository.mongo.providers.DataStoreConfCodecProviders$DataStoreConfCodecProvider$.encodeWithType(DataStoreConfCodecProviders.scala:28)
at it.agilelab.bigdata.wasp.repository.mongo.providers.DataStoreConfCodecProviders$DataStoreConfCodecProvider$.encodeWithType(DataStoreConfCodecProviders.scala:13)
at it.agilelab.bigdata.wasp.repository.mongo.providers.SealedTraitCodecProvider$$anon$1.encode(SealedTraitCodecProvider.scala:22)
at it.agilelab.bigdata.wasp.repository.mongo.providers.AbstractCodecProvider$$anonfun$writeList$1.apply(AbstractCodecProvider.scala:69)
at it.agilelab.bigdata.wasp.repository.mongo.providers.AbstractCodecProvider$$anonfun$writeList$1.apply(AbstractCodecProvider.scala:68)
at scala.collection.immutable.List.foreach(List.scala:392)
at it.agilelab.bigdata.wasp.repository.mongo.providers.AbstractCodecProvider.writeList(AbstractCodecProvider.scala:68)
at it.agilelab.bigdata.wasp.repository.mongo.providers.BatchGdprETLModelCodecProvider$.encodeClass(BatchETLCodecProvider.scala:66)
at it.agilelab.bigdata.wasp.repository.mongo.providers.BatchGdprETLModelCodecProvider$.encodeClass(BatchETLCodecProvider.scala:41)
at it.agilelab.bigdata.wasp.repository.mongo.providers.AbstractCodecProvider$$anon$1.encode(AbstractCodecProvider.scala:27)
at org.bson.codecs.EncoderContext.encodeWithChildContext(EncoderContext.java:91)
at org.mongodb.scala.bson.codecs.macrocodecs.MacroCodec$class.writeValue(MacroCodec.scala:168)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$BatchJobDBModelV1MacroCodec$3.writeValue(VersionedProviders.scala:139)
at org.mongodb.scala.bson.codecs.macrocodecs.MacroCodec$class.writeFieldValue(MacroCodec.scala:159)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$BatchJobDBModelV1MacroCodec$3.writeFieldValue(VersionedProviders.scala:139)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$BatchJobDBModelV1MacroCodec$3.writeCaseClassData(VersionedProviders.scala:139)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$BatchJobDBModelV1MacroCodec$3.writeCaseClassData(VersionedProviders.scala:139)
at org.mongodb.scala.bson.codecs.macrocodecs.MacroCodec$class.writeValue(MacroCodec.scala:165)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$BatchJobDBModelV1MacroCodec$3.writeValue(VersionedProviders.scala:139)
at org.mongodb.scala.bson.codecs.macrocodecs.MacroCodec$class.encode(MacroCodec.scala:92)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedRegistry$BatchJobDBModelV1MacroCodec$3.encode(VersionedProviders.scala:139)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedCodecProvider.createBsonDocument(VersionedCodecProvider.scala:78)
at it.agilelab.bigdata.wasp.repository.mongo.providers.VersionedCodecProvider$$anon$1.encode(VersionedCodecProvider.scala:64)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29)
at com.mongodb.operation.BulkWriteBatch$WriteRequestEncoder.encode(BulkWriteBatch.java:403)
at com.mongodb.operation.BulkWriteBatch$WriteRequestEncoder.encode(BulkWriteBatch.java:375)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63)
at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29)
at com.mongodb.internal.connection.BsonWriterHelper.writeDocument(BsonWriterHelper.java:77)
at com.mongodb.internal.connection.BsonWriterHelper.writePayload(BsonWriterHelper.java:59)
at com.mongodb.internal.connection.CommandMessage.encodeMessageBodyWithMetadata(CommandMessage.java:147)
at com.mongodb.internal.connection.RequestMessage.encode(RequestMessage.java:138)
at com.mongodb.internal.connection.CommandMessage.encode(CommandMessage.java:61)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceiveAsync(InternalStreamConnection.java:329)
at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceiveAsync(UsageTrackingInternalConnection.java:114)
at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceiveAsync(DefaultConnectionPool.java:461)
at com.mongodb.internal.connection.CommandProtocolImpl.executeAsync(CommandProtocolImpl.java:78)
at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.executeAsync(DefaultServer.java:241)
at com.mongodb.internal.connection.DefaultServerConnection.executeProtocolAsync(DefaultServerConnection.java:285)
at com.mongodb.internal.connection.DefaultServerConnection.commandAsync(DefaultServerConnection.java:156)
at com.mongodb.operation.MixedBulkWriteOperation.executeCommandAsync(MixedBulkWriteOperation.java:444)
at com.mongodb.operation.MixedBulkWriteOperation.executeBatchesAsync(MixedBulkWriteOperation.java:350)
at com.mongodb.operation.MixedBulkWriteOperation.access$1000(MixedBulkWriteOperation.java:72)
at com.mongodb.operation.MixedBulkWriteOperation$2$1.call(MixedBulkWriteOperation.java:239)
at com.mongodb.operation.OperationHelper.validateWriteRequests(OperationHelper.java:189)
at com.mongodb.operation.MixedBulkWriteOperation$2.call(MixedBulkWriteOperation.java:222)
at com.mongodb.operation.OperationHelper$7.onResult(OperationHelper.java:614)
at com.mongodb.operation.OperationHelper$7.onResult(OperationHelper.java:611)
at com.mongodb.internal.connection.DefaultServer$1.onResult(DefaultServer.java:116)
at com.mongodb.internal.connection.DefaultServer$1.onResult(DefaultServer.java:105)
at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:49)
at com.mongodb.internal.connection.DefaultConnectionPool.openAsync(DefaultConnectionPool.java:201)
at com.mongodb.internal.connection.DefaultConnectionPool.getAsync(DefaultConnectionPool.java:158)
at com.mongodb.internal.connection.DefaultServer.getConnectionAsync(DefaultServer.java:105)
at com.mongodb.binding.AsyncClusterBinding$AsyncClusterBindingConnectionSource.getConnection(AsyncClusterBinding.java:139)
at com.mongodb.operation.OperationHelper.withAsyncConnectionSource(OperationHelper.java:611)
at com.mongodb.operation.OperationHelper.access$200(OperationHelper.java:63)
at com.mongodb.operation.OperationHelper$AsyncCallableWithConnectionAndSourceCallback.onResult(OperationHelper.java:631)
at com.mongodb.operation.OperationHelper$AsyncCallableWithConnectionAndSourceCallback.onResult(OperationHelper.java:619)
at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:49)
at com.mongodb.binding.AsyncClusterBinding$1.onResult(AsyncClusterBinding.java:113)
at com.mongodb.binding.AsyncClusterBinding$1.onResult(AsyncClusterBinding.java:107)
at com.mongodb.internal.connection.BaseCluster$ServerSelectionRequest.onResult(BaseCluster.java:440)
at com.mongodb.internal.connection.BaseCluster.handleServerSelectionRequest(BaseCluster.java:304)
at com.mongodb.internal.connection.BaseCluster.selectServerAsync(BaseCluster.java:160)
at com.mongodb.internal.connection.SingleServerCluster.selectServerAsync(SingleServerCluster.java:41)
at com.mongodb.binding.AsyncClusterBinding.getAsyncClusterBindingConnectionSource(AsyncClusterBinding.java:107)
at com.mongodb.binding.AsyncClusterBinding.getWriteConnectionSource(AsyncClusterBinding.java:102)
at com.mongodb.operation.OperationHelper.withAsyncConnection(OperationHelper.java:550)
at com.mongodb.operation.MixedBulkWriteOperation.executeAsync(MixedBulkWriteOperation.java:216)
at com.mongodb.async.client.OperationExecutorImpl$2$1.onResult(OperationExecutorImpl.java:134)
at com.mongodb.async.client.OperationExecutorImpl$2$1.onResult(OperationExecutorImpl.java:128)
at com.mongodb.async.client.OperationExecutorImpl.getReadWriteBinding(OperationExecutorImpl.java:186)
at com.mongodb.async.client.OperationExecutorImpl.access$200(OperationExecutorImpl.java:45)
at com.mongodb.async.client.OperationExecutorImpl$2.onResult(OperationExecutorImpl.java:126)
at com.mongodb.async.client.OperationExecutorImpl$2.onResult(OperationExecutorImpl.java:120)
at com.mongodb.async.client.ClientSessionHelper$2.onResult(ClientSessionHelper.java:80)
at com.mongodb.async.client.ClientSessionHelper$2.onResult(ClientSessionHelper.java:73)
at com.mongodb.internal.connection.BaseCluster$ServerSelectionRequest.onResult(BaseCluster.java:440)
at com.mongodb.internal.connection.BaseCluster.handleServerSelectionRequest(BaseCluster.java:304)
at com.mongodb.internal.connection.BaseCluster.selectServerAsync(BaseCluster.java:160)
at com.mongodb.internal.connection.SingleServerCluster.selectServerAsync(SingleServerCluster.java:41)
at com.mongodb.async.client.ClientSessionHelper.createClientSession(ClientSessionHelper.java:68)
at com.mongodb.async.client.ClientSessionHelper.withClientSession(ClientSessionHelper.java:51)
at com.mongodb.async.client.OperationExecutorImpl.execute(OperationExecutorImpl.java:120)
at com.mongodb.async.client.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:1127)
at com.mongodb.async.client.MongoCollectionImpl.executeReplaceOne(MongoCollectionImpl.java:645)
at com.mongodb.async.client.MongoCollectionImpl.replaceOne(MongoCollectionImpl.java:620)
at com.mongodb.async.client.MongoCollectionImpl.replaceOne(MongoCollectionImpl.java:614)
at org.mongodb.scala.MongoCollection$$anonfun$replaceOne$2.apply(MongoCollection.scala:802)
at org.mongodb.scala.MongoCollection$$anonfun$replaceOne$2.apply(MongoCollection.scala:802)
at org.mongodb.scala.internal.ObservableHelper$$anon$1.apply(ObservableHelper.scala:35)
at org.mongodb.scala.internal.ObservableHelper$$anon$1.apply(ObservableHelper.scala:34)
at com.mongodb.async.client.SingleResultCallbackSubscription.requestInitialData(SingleResultCallbackSubscription.java:38)
at com.mongodb.async.client.AbstractSubscription.tryRequestInitialData(AbstractSubscription.java:164)
at com.mongodb.async.client.AbstractSubscription.request(AbstractSubscription.java:87)
at org.mongodb.scala.ObservableImplicits$BoxedSubscription.request(ObservableImplicits.scala:493)
at org.mongodb.scala.ObservableImplicits$ScalaObservable$$anon$2.onSubscribe(ObservableImplicits.scala:366)
at org.mongodb.scala.ObservableImplicits$ToSingleObservable$$anon$3.onSubscribe(ObservableImplicits.scala:441)
at org.mongodb.scala.internal.SubscriptionCheckingObserver.onSubscribe(SubscriptionCheckingObserver.scala:28)
at org.mongodb.scala.Observer$class.onSubscribe(Observer.scala:85)
at org.mongodb.scala.internal.SubscriptionCheckingObserver.onSubscribe(SubscriptionCheckingObserver.scala:21)
at com.mongodb.async.client.SingleResultCallbackSubscription.(SingleResultCallbackSubscription.java:33)
at com.mongodb.async.client.Observables$2.subscribe(Observables.java:78)
at org.mongodb.scala.ObservableImplicits$BoxedObservable.subscribe(ObservableImplicits.scala:477)
at org.mongodb.scala.ObservableImplicits$ToSingleObservable.subscribe(ObservableImplicits.scala:432)
at org.mongodb.scala.ObservableImplicits$ScalaObservable.headOption(ObservableImplicits.scala:358)
at org.mongodb.scala.ObservableImplicits$ScalaObservable.head(ObservableImplicits.scala:344)
at it.agilelab.bigdata.wasp.repository.mongo.utils.MongoDBHelper$ImplicitObservable$class.headResult(MongoDBHelper.scala:159)
at it.agilelab.bigdata.wasp.repository.mongo.utils.MongoDBHelper$GenericObservable.headResult(MongoDBHelper.scala:150)
at it.agilelab.bigdata.wasp.repository.mongo.utils.MongoDBHelper$class.replaceDocumentToCollection(MongoDBHelper.scala:129)
at it.agilelab.bigdata.wasp.repository.mongo.WaspDBMongoImp.replaceDocumentToCollection(WaspMongoDB.scala:122)
at it.agilelab.bigdata.wasp.repository.mongo.WaspDBMongoImp.upsert(WaspMongoDB.scala:272)
at it.agilelab.bigdata.wasp.repository.mongo.bl.BatchJobBLImp.upsert(BatchJobBLImp.scala:32)
at com.vodafone.automotive.wasp.master.launcher.MasterNodeLauncher$.launch(MasterNodeLauncher.scala:146)
at it.agilelab.bigdata.wasp.core.launcher.WaspLauncher$class.main(WaspLauncher.scala:55)
at com.vodafone.automotive.wasp.master.launcher.MasterNodeLauncher$.main(MasterNodeLauncher.scala:82)
at com.vodafone.automotive.wasp.master.launcher.MasterNodeLauncher.main(MasterNodeLauncher.scala)

Answer 1 · 2021-11-22T14:08:39.000Z

Thanks for the report @davidcoste , we are looking into it.

Answer 2 · 2021-11-22T18:40:25.000Z

Hi @davidcoste would you mind trying out the latest snapshot version 2.27.0-cdp717-SNAPSHOT?

Answer 3 · 2021-11-23T09:05:08.000Z

Hi @tmnd1991
Is it available on a repository ?

Answer 4 · 2021-11-23T10:00:18.000Z

Hi, I digged into it and I tested a fix, it is available on this version 2.27.0-520-gdpr-deletion-job-upsert-on-waspdb-fails-for-mongorepository-cdp717-SNAPSHOT not the one I mentioned earlier.
Can you try it out and confirm it works?
If sbt can't find the dependency, be sure to have between your resolvers also sonatype snapshots Resolver.sonatypeRepo("snapshots")

Answer 5 · 2021-11-23T10:00:42.000Z

the artefact is being currently published so it might be available in 10 minutes or so

Answer 6 · 2021-11-23T10:26:56.000Z

it is available, let us know if that fixes your issue so that we can perform an hot fix release

https://oss.sonatype.org/content/repositories/snapshots/it/agilelab/

Answer 7 · 2021-11-23T13:31:06.000Z

I still can't get it from the repository:
For exemple: https://oss.sonatype.org/content/repositories/snapshots/it/agilelab/wasp-plugin-hbase-spark_2.11/2.27.0-cdp717-SNAPSHOT/wasp-plugin-hbase-spark_2.11-2.27.0-cdp717-SNAPSHOT.pom returns an http 404

Answer 8 · 2021-11-23T13:48:19.000Z

Hi, I digged into it and I tested a fix, it is available on this version 2.27.0-520-gdpr-deletion-job-upsert-on-waspdb-fails-for-mongorepository-cdp717-SNAPSHOT not the one I mentioned earlier.

OK, sorry I did not see this comment

Answer 9 · 2021-11-23T14:47:25.000Z

@tmnd1991 ,
I confirm that the fix is working.
Thanks for your quick action

Answer 10 · 2021-11-23T14:54:23.000Z

Cool, I will proceed to release 2.27.1 with this fix.