Using for complex hbase format
pranny opened this issue · 1 comments
pranny commented
I have a hbase table with multiple column families. From the given example, I am not sure how do I create my configs.
My table looks like
column family: main, column name: value
column family: main, column name: timestamp
column family: meta, column name: datatype
The Kafka Topic has Value schema that goes like
fields:
- key: Corresponds to the hbase key
- value: Corresponds to the "main:value"
- timestamp: Corresponds to "main:timestamp"
- datatype: Corresponds to "meta:datatype"
Is something like below a valid config?
tasks.max=1
topics=telemetry
zookeeper.quorum=localhost:2181
event.parser.class=io.svectors.hbase.parser.AvroEventParser
hbase.telemetry.rowkey.columns=value, timestamp, datatype
hbase.telemetry.rowkey.delimiter=:
hbase.telemetry.family=main, meta
nishutayal commented
It seems, it supports one column family as of now. I have enhanced the code with multiple column family support and custom column mapping. You can refer this for your use case:
https://github.com/nishutayal/kafka-connect-hbase