got "error: The JSON element does not have the requested type" while following the guide to load data
Opened this issue · 0 comments
When I followed the Kafka-connector-starrocks guide to load data to StarRocks, I got the error message below.
ERROR [starrocks-kafka-connector|task-0] catch exception, wait rollback (com.starrocks.data.load.stream.v2.StreamLoadManagerV2:424)
com.starrocks.data.load.stream.exception.StreamLoadFailException: Transaction prepare failed, db: example_db, table: test-sqlite-jdbc-accounts, label: -0aba9ed9-fd83-41c3-afcf-9be12bbc926f,
responseBody: {
"Status": "DATA_QUALITY_ERROR",
"Message": "Failed to iterate document stream as object. error: The JSON element does not have the requested type.",
"ErrorURL": "http://10.5.0.3:8040/api/_load_error_log?file=error_log_174479fdeb882113_29549abd35d3bdb2"
}
errorLog: Error: Data quality error: Failed to iterate document stream as object. error: The JSON element does not have the requested type.. Row: parser current location: [{"id":19,"name":"cathy"}]
at com.starrocks.data.load.stream.TransactionStreamLoader.prepare(TransactionStreamLoader.java:221)
at com.starrocks.data.load.stream.v2.TransactionTableRegion.commit(TransactionTableRegion.java:247)
at com.starrocks.data.load.stream.v2.StreamLoadManagerV2.lambda$init$0(StreamLoadManagerV2.java:210)
at java.lang.Thread.run(Thread.java:750)
I can get the same error message if I run the command below.
curl --location-trusted -u root \
-d '[{"id":19,"name":"cathy"}]' \
-H "label:test-3" \
-H "format: JSON" \
-H "columns:id,name" \
-H "Expect:100-continue" \
-XPUT http://localhost:8030/api/example_db/test-sqlite-jdbc-accounts/_stream_load
But if I add a header by -H "strip_outer_array:true"
, it will succeed.
So I fix the error by adding "sink.properties.strip_outer_array=true" to the properties file of Kafka-connector-starrocks plugin, because the configuration item will cause kafka-connector-starrocks adding "strip_outer_array:true" to http request header when do stream load.
My question is, is it possible to add the "strip_outer_array:true" header automatically by kafka-connector-starrocks, or do not wrap the data with '[ ]'?
BTW, I submitted a PR to modify the guide by adding "sink.properties.strip_outer_array=true" to the properties file.
StarRocks/starrocks#41218