Camel Kafka Connector

Chat on Gitter Master Build

Introduction

Note

The project is WIP!

This is a "Camel Kafka connector adapter" that aims to provide a user friendly way to use all Apache Camel components in Kafka Connect. For more information about Kafka Connect take a look here.

Build the project

mvn clean package

Build the project and run integration tests

To build the project it is sufficient to:

mvn clean install

To run the integration tests it is required to:

  • have Docker version 17.05 or higher running

  • run:

    mvn -DskipIntegrationTests=false clean verify package

It is also possible to point the tests to use an external services. To do so, you must set properties for the services that you want to run. This causes the tests to not launch the local container and use existing remote instances. At the moment, the following properties can be set for remote testing:

  • kafka.instance.type

    • kafka.bootstrap.servers

  • aws-service.instance.type

    • access.key: AWS access key (mandatory for remote testing)

    • secret.key: AWS secret key (mandatory for remote testing)

    • aws.region: AWS region (optional)

    • aws.host: AWS host (optional)

  • aws-service.kinesis.instance.type

    • access.key: AWS access key (mandatory for remote testing)

    • secret.key: AWS secret key (mandatory for remote testing)

    • aws.region: AWS region (optional)

    • aws.host: AWS host (optional)

  • elasticsearch.instance.type

    • elasticsearch.host

    • elasticsearch.port

  • cassandra.instance.type

    • cassandra.host

    • cassandra.cql3.port

  • jms-service.instance.type

    • jms.broker.address

mvn -Dkafka.bootstrap.servers=host1:port -Dkafka.instance.type=remote -DskipIntegrationTests=false clean verify package

It’s possible to use a properties file to set these properties. To do so use -Dtest.properties=/path/to/file.properties.

Try it out locally

You can use Camel Kafka connectors with local Apache Kafka installation. For more details see the local guide.

Try it out on OpenShift with Strimzi

You can also use Camel Kafka connectors on OpenShift. For more details see the OpenShift guide.

Known limitations

In order to use other Camel components you need to add them as dependencies in the project pom.xml.