ClickHouse/clickhouse-kafka-connect

Receiving "ClickHouseException: std::exception. Code: 1001, type: std::__1::bad_function_call" in logs, connector does not work

mbkh68 opened this issue · 3 comments

mbkh68 commented

Receiving this error when trying to setup clickhouse sink connector, no record is saved into database.

org.apache.kafka.connect.errors.RetriableException: java.lang.RuntimeException: Topic: [mytopic], Partition: [0], MinOffset: [25312109], MaxOffset: [25312160], (QueryId: [77d6245c-5f63-4881-8311-973b26c7b846])
 at com.clickhouse.kafka.connect.util.Utils.handleException(Utils.java:83)
 at com.clickhouse.kafka.connect.sink.ClickHouseSinkTask.put(ClickHouseSinkTask.java:68)
 at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:586)
 at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:329)
 at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:232)
 at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:201)
 at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:185)
 at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:234)
 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
 at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.RuntimeException: Topic: [mytopic], Partition: [0], MinOffset: [25312109], MaxOffset: [25312160], (QueryId: [77d6245c-5f63-4881-8311-973b26c7b846])
 at com.clickhouse.kafka.connect.sink.processing.Processing.doInsert(Processing.java:63)
 at com.clickhouse.kafka.connect.sink.processing.Processing.doLogic(Processing.java:138)
 at com.clickhouse.kafka.connect.sink.ProxySinkTask.put(ProxySinkTask.java:99)
 at com.clickhouse.kafka.connect.sink.ClickHouseSinkTask.put(ClickHouseSinkTask.java:64)
 ... 11 more
Caused by: java.util.concurrent.ExecutionException: com.clickhouse.client.ClickHouseException: std::exception. Code: 1001, type: std::__1::bad_function_call, e.what() = std::bad_function_call (version 24.1.2.5 (official build))
, server ClickHouseNode [uri=https://clickhouse-clickhouse.clickhouse:8443/default, options={sslmode=none}]@-2026445492
 at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
 at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
 at com.clickhouse.kafka.connect.sink.db.ClickHouseWriter.doInsertJson(ClickHouseWriter.java:659)
 at com.clickhouse.kafka.connect.sink.db.ClickHouseWriter.doInsert(ClickHouseWriter.java:171)
 at com.clickhouse.kafka.connect.sink.processing.Processing.doInsert(Processing.java:61)
 ... 14 more
Caused by: com.clickhouse.client.ClickHouseException: std::exception. Code: 1001, type: std::__1::bad_function_call, e.what() = std::bad_function_call (version 24.1.2.5 (official build))
, server ClickHouseNode [uri=https://clickhouse-clickhouse.clickhouse:8443/default, options={sslmode=none}]@-2026445492
 at com.clickhouse.client.ClickHouseException.of(ClickHouseException.java:168)
 at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:275)
 at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
 ... 3 more
Caused by: java.io.IOException: std::exception. Code: 1001, type: std::__1::bad_function_call, e.what() = std::bad_function_call (version 24.1.2.5 (official build))
 at com.clickhouse.client.http.ApacheHttpConnectionImpl.checkResponse(ApacheHttpConnectionImpl.java:209)
 at com.clickhouse.client.http.ApacheHttpConnectionImpl.post(ApacheHttpConnectionImpl.java:243)
 at com.clickhouse.client.http.ClickHouseHttpClient.send(ClickHouseHttpClient.java:118)
 at com.clickhouse.client.AbstractClient.sendAsync(AbstractClient.java:161)
 at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:273)

Configuration

{
    "name": "clickhouse-sink-connector",
    "config": {
        "connector.class": "com.clickhouse.kafka.connect.ClickHouseSinkConnector",
        "tasks.max": 1,        
        "topics": "mytopic",
        "ssl": true,
        "hostname": "clickhouse-clickhouse.clickhouse",
        "database": "default",
        "password": "password",
        "port": 8443,
        "jdbcConnectionProperties": "?ssl=true&sslmode=none",
        "value.converter.schemas.enable": false,
        "value.converter": "org.apache.kafka.connect.json.JsonConverter",
        "timeoutSeconds": 30,
        "insertFormat": "JSON",
        "exactlyOnce": true,
        "username": "admin"
    }
}

Environment

  • I have built connector from source from this PR (#328) and created my own docker image.
  • Kafka-Connect version: 2.7.0
  • Kafka version: 2.7.0
  • Kafka environment: strimzi
  • OS: EKS

ClickHouse server

  • ClickHouse Server version: 24.1.2.5
mbkh68 commented

After removing table columns of type datetime issue resolved, how can we have custom formatting for datetime columns?

mbkh68 commented

I think need to have better error handling for this case, spend half of my day finding this!

  • 👀1
mbkh68 commented

Finally got it fixed by adding this field to connector config:
"clickhouseSettings": "date_time_input_format=best_effort"

Closing

  • 👍2