/kafka-connect-log-analytics

Primary LanguageJavaApache License 2.0Apache-2.0

kafka-connect-log-analytics

Log analytics Sink Connector for Kafka-connect

  • this connector pull messages from Kafka topic and send to log analytics custom logs
  • Only supported message format is JSON with schema, make sure valid json is present in Kafka topic
  • Batch 100 records at a time for a given poll cycle
  • This derives log analytics columns based on the schema of the message received
  • Messages appear in log analytics under "Custom Logs" with "_CL" as post fix to given topic i.e. if "DATASINKLOGS" is source topic then messages will be displayed in log analytics under custom logs with name "DATASINKLOGS_CL"

Connector Definition

{
“name”:”sink-connector-1”
"connector.class": "io.kafka.connect.log.anlaytics.sink.LogAnlayticsSinkTask",
"tasks.max": "1",
"workspace.id":"xxxxx-xxxx-xxxx-xxxx-xxxxxxxx",
"workspace.key":"xxxxxxxxxx==",
"topics": "DATASINKLOGS"
}