confluentinc/kafka-connect-datagen

NPE if `schema.keyfield` is not present in schema

Opened this issue · 4 comments

rmoff commented

If the connector is launched with a config for in which the field schema.keyfield specified is not present in the schema, the connector aborts with NPE

[2019-06-11 09:46:20,940] ERROR WorkerSourceTask{id=source-datagen-orders-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask)
java.lang.NullPointerException
   at io.confluent.kafka.connect.datagen.DatagenTask.poll(DatagenTask.java:181)
   at org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:245)
   at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:221)
   at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
   at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at java.lang.Thread.run(Thread.java:748)
[2019-06-11 09:46:20,941] ERROR WorkerSourceTask{id=source-datagen-orders-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)

The connector should validate the field from the schema and if missing throw a proper error to the user.

ybyzek commented

@rmoff thanks for catching and reporting the issue. Let's use this GH issue to track if other users encounter it.

aivarssilins commented

Yes, same issue. Is tat field mandatory? Also how to specify if need to get that from nested structure? Put like headers.eventId

  • 👍2
josephwibowo commented

no update since 2019? I have the same issue

yashmayya commented

The NPE bug has been fixed in version 0.5.0 of the connector (https://www.confluent.io/hub/confluentinc/kafka-connect-datagen) with the addition of schema validations (#84). A bugfix has also been added in 0.4.1 where the connector will throw a more appropriate exception if the key is missing rather than an NPE (#83)