This is source code for tutorial of connecting kafka's streaming event to Machbase. It is based on edenhill/librdkafka library, including Machbase's C connector. You can freely use with or without tutorial page at link here.
export MACHBASE_IP='localhost'
export MACHBASE_PORT_NO=5656
If you don't set those values, then default IP/Port will be affected. (127.0.0.1/5656)
In addition to that, you should check if $MACHBASE_HOME
is set.
Please follow install instructions via documentation. (If you want to read Korean manual, please follow here.)
Please follow Kafka's quickstart page. Then, remember Kafka installation path.
After cloning this repo, go inside and type;
make all
Using machsql, run the CREATE TABLE
query written in query/create.sql
.
At repository directory,
machsql -f query/create.sql
At repository directory,
python gen.py ${COUNT} > /path/to/kafka/install/tag.csv
${COUNT}
is the number of tag data that you want to populate.
At kafka's directory,
$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic tag
If you install and startup zookeeper server at other IP/Port, you should change connection information described above.
At repository directory,
./kafka_to_machbase tag
It waits further data to consume from the kafka..
At kafka's directory,
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic tag < tag.csv
If you install and startup kafka broker at other IP/Port, you should change connection information described above.
It produces tag data written in tag.csv, and the consumer will retrieve and append to Machbase server.
Using machsql, run SELECT
query to the tagdata table.
SELECT COUNT(*) FROM TAG;