emqx_kafka_bridge

This is a plugin for the EMQX broker that sends all messages received by the broker to kafka.

注意：插件目前适配的是emqx 3.0，不适配3.1,3.2这些版本了

Build the EMQX broker

Clone emqx-relx project

We need to clone the EMQX project GITHUB

  git clone https://github.com/emqx/emqx-rel.git

Add EMQ Kafka bridge as a DEPS Adding EMQ kafka bridge as a dependency in the Makefile.
1. search for DEPS += $(foreach dep,$(OUR_APPS),$(call app_name,$(dep))) add the following line before the above lines
  
  DEPS += emqx_kafka_bridge
2. search for
```
$(foreach dep,$(OUR_APPS),$(eval dep_$(call app_name,$(dep)) = git-emqx https://github.com/emqx/$(dep) $(call app_vsn,$(dep))))
```
add the following line before the above lines

dep_emqx_kafka_bridge = git https://github.com/bob403/emqx_kafka_bridge.git master
Add load plugin in relx.config

{emqx_kafka_bridge, load},
Build
```
cd emqx-relx && make
```

Configuration

You will have to edit the configurations of the bridge to set the kafka Ip address and port.

Edit the file emq-relx/deps/emqx_kafka_bridge/etc/emqx_kafka_bridge.conf

##--------------------------------------------------------------------
## kafka Bridge
##--------------------------------------------------------------------

## The Kafka loadbalancer node host that bridge is listening on.
##
## Value: 127.0.0.1, localhost
kafka.host = 127.0.0.1

## The kafka loadbalancer node port that bridge is listening on.
##
## Value: Port
kafka.port = 9092

## The kafka loadbalancer node partition strategy.
##
## Value: strict_round_robin
kafka.partitionstrategy = strict_round_robin

## payload topic.
##
## Value: string
kafka.payloadtopic = Processing

Start the EMQ broker and load the plugin

cd emqx-relx/_rel/emqx
./bin/emqx start
./bin/emqx_ctl plugins load emqx_kafka_bridge

Test

Send a MQTT message on a random topic from a MQTT client to your EMQ broker.

The following should be received by your kafka consumer :

{"topic":"yourtopic", "message":[yourmessage]} This is the format in which kafka will receive the MQTT messages

If Kafka consumer shows no messages even after publishing to EMQX - ACL makes the plugin fail, so please remove all the ACL related code to ensure it runs properly. We will soon push the updated (Working) code to the repository.

License