mmolimar/kafka-connect-fs

Source connector handling formatted JSON type records in different way

Opened this issue · 1 comments

Hi

Can source connector handle formatted JSON records. I have file contains JSON records like this

{
"Employee":{
"header":{
"username":"abc",
"reference":"1d0d5113-3d22-44c0-99f5-f6ff642f0233"
},
"payload":{
"name":"hello",
"sal":""
}
}
}
{
"Employee":{
"header":{
"username":"abc",
"reference":"1d0d5113-3d22-44c0-99f5-f6ff642f0299"
},
"payload":{
"name":"hello",
"sal":"10000"
}
}
}

It is considering each line as new record i.e. here 24 records and what I found out that it supports only minify JSON.
Please let me know if my understanding is correct. ?

Note : I am using latest version i.e. v1.2.0 and it is processing correctly if my files contains JSON with below style:

{"Employee":{"header":{"username":"abc","reference":"1d0d5113-3d22-44c0-99f5-f6ff642f0233"},"payload":{"name":"hello","sal":""}}}
{"Employee":{"header":{"username":"abc","reference":"1d0d5113-3d22-44c0-99f5-f6ff642f0299"},"payload":{"name":"hello","sal":"10000"}}}

Regards
Deepak

Hi!
You can set the property file_reader.json.record_per_line to configure this.
All config options are in the documentation here.