Fully automated Kafka and Confluent Docker based examples // Easily build reproduction models
π₯ Check out kafka-docker-playground.io to learn how to use the playground, how it works and how to build your own reproduction models !
- ActiveMQ Sink (also with π mTLS)
- ActiveMQ Source (also with π mTLS)
- Amazon CloudWatch Logs Source
- Amazon CloudWatch Metrics Sink
- Amazon DynamoDB Sink (also with π proxy)
- Amazon Kinesis Source (also with π proxy)
- Amazon Redshift Sink
- Amazon Redshift Source (using JDBC)
- Amazon S3 Sink (also with π proxy)
- Amazon S3 Source
- Amazon SQS Source (also with π proxy)
- AMPS Source
- Apache Kudu Source
- Apache Kudu Sink
- AWS Lambda Sink
- Azure Blob Storage Sink (also with π proxy)
- Azure Blob Storage Source (also with π proxy)
- Azure Cosmos DB Sink
- Azure Cosmos DB Source
- Azure Data Lake Storage Gen1 Sink
- Azure Data Lake Storage Gen2 Sink
- Azure Event Hubs Source
- Azure Functions Sink
- Azure Cognitive Search Sink (also with π proxy)
- Azure Service Bus Source
- Azure SQL Data Warehouse Sink
- Cassandra Sink
- Couchbase Sink
- Couchbase Source
- Debezium CDC Microsoft SQL Server Source
- Debezium CDC MySQL Source
- Debezium CDC PostgreSQL Source (also with π SSL and mTLS)
- Debezium CDC MongoDB Source
- Data Diode Sink
- Data Diode Source
- Datadog Metrics Sink
- ElasticSearch Sink
- FilePulse Source
- FileStream Source
- FileStream Sink
- FTPS Sink
- FTPS Source
- Gemfire Sink
- Github Source
- Google BigQuery Sink
- Google Cloud BigTable Sink (also with π proxy)
- Google Cloud Functions Sink
- Google Cloud Pub/Sub Source (also with π proxy)
- Google Cloud Spanner Sink (also with π proxy)
- Google Cloud Storage Sink
- Google Cloud Storage Source
- Google Firebase Realtime Database Sink
- Google Firebase Realtime Database Source
- HBase Sink
- HDFS 2 Source
- HDFS 3 Source
- HDFS 2 Sink
- HDFS 3 Sink
- HTTP Sink
- IBM MQ Sink (also with π SSL and mTLS)
- IBM MQ Source (also with π SSL and mTLS)
- InfluxDB Sink
- InfluxDB Source
- JDBC CockroachDB Source
- JDBC IBM DB2 Sink
- JDBC IBM DB2 Source
- JDBC MySQL Sink (also with π SSL and mTLS)
- JDBC Oracle 11 Sink
- JDBC Oracle 12 Sink (also with π SSL and mTLS)
- JDBC Oracle 19c Sink (also with π SSL and mTLS)
- JDBC PostGreSQL Sink (also with π SSL and mTLS)
- JDBC Microsoft SQL Server Sink (also with π SSL)
- JDBC MySQL Source (also with π SSL and mTLS)
- JDBC Oracle 11 Source
- JDBC Oracle 12 Source (also with π SSL and mTLS)
- JDBC Oracle 19c Source (also with π SSL and mTLS)
- JDBC PostGreSQL Source (also with π SSL and mTLS)
- JDBC Microsoft SQL Server Source (also with π SSL)
- JDBC Snowflake Sink
- JDBC Snowflake Source
- JDBC Vertica Sink
- JIRA Source
- JMS ActiveMQ Sink
- JMS Solace Sink
- JMS TIBCO EMS Sink
- JMS TIBCO EMS Source
- JMS Oracle Weblogic Sink
- JMS Oracle Weblogic Source
- Mapr Sink
- Marketo Source (also with π proxy)
- Minio Sink
- MongoDB Sink
- MongoDB Source
- MQTT Sink
- MQTT Source
- Neo4j Sink
- OmniSci Sink
- Oracle 11 CDC Source
- Oracle 12 CDC Source (also with π SSL and mTLS)
- Oracle 19c CDC Source (also with π SSL and mTLS)
- Oracle Weblogic Source
- PagerDuty Sink (also with π proxy)
- Prometheus Sink
- RabbitMQ Sink
- RabbitMQ Source
- Redis Sink
- SalesForce Bulk API Sink (also with π proxy)
- SalesForce Bulk API Source (also with π proxy)
- SalesForce CDC Source (also with π proxy)
- SalesForce Platform Events Sink (also with π proxy)
- SalesForce Platform Events Source (also with π proxy)
- SalesForce PushTopics Source (also with π proxy)
- SalesForce SObject Sink (also with π proxy)
- ServiceNow Sink (also with π proxy)
- ServiceNow Source (also with π proxy)
- SFTP Sink (also with π Kerberos)
- SFTP Source (also with π Kerberos)
- SNMP Trap Source
- Snowflake Sink (also with π proxy)
- Solace Sink
- Solace Source
- Splunk Sink
- Splunk Source
- Splunk S2S Source
- Spool Dir Source
- Syslog Source
- TIBCO EMS Sink
- TIBCO EMS Source
- Vertica Sink
- Zendesk Source
* You can change default connector version by setting CONNECTOR_TAG
environment variable before starting a test, get more details here
- π How to connect your components to Confluent Cloud
- π How to monitor your Confluent Cloud cluster using Metrics API and Grafana, Splunk, New Relic, etc...
- AWS Kinesis source
- ServiceNow source
- ServiceNow sink
- MongoDB source
- MQTT Source
- Firebase
- βοΈ Using cp-ansible with Confluent Cloud
- π€ Using cp-helm-charts with Confluent Cloud
- π¦οΈ Using Confluent Operator with Confluent Cloud
- π¨ Using Confluent for Kubenetes with Confluent Cloud
- π Demo using dabz/ccloudexporter
- π .NET client (producer/consumer)
- π― Go client (producer/consumer)
- π KafkaJS client (producer/consumer)
- π Python client (producer/consumer)
- π§βπ Managing topics and ACLs using matt-mangia/kafka-admin
- β»οΈ Confluent Replicator OnPrem to cloud and Cloud to Cloud examples
- πΊοΈ Multi-Cluster Schema Registry with hybrid configuration (onprem/confluent cloud)
- π Confluent REST Proxy Security Plugin with Principal Propagation
- ποΈ Confluent Schema Registry Security Plugin
- π¦ Migrate Schemas to Confluent Cloud using Confluent Replicator
- π° Confluent Cloud Networking using HAProxy
- ποΈ Apache Mirror Maker 2
- β»οΈ Cluster Linking Quick Start with service account only
Using Multi-Data-Center setup with πΊπΈ and πͺπΊ clusters
- π Confluent Replicator as connector
- With PLAINTEXT
- With SASL_PLAIN
- With Kerberos
- ποΈ Confluent Replicator as executable
- With PLAINTEXT
- With SASL_PLAIN
- With Kerberos
- ποΈ Apache Mirror Maker 2
- With PLAINTEXT
- β»οΈ Cluster Linking
Using single cluster:
- PLAINTEXT: no security
- SASL/PLAIN: no SSL encryption, SASL/PLAIN authentication
- SASL/SCRAM no SSL encryption, SASL/SCRAM-SHA-256 authentication
- SASL/SSL: SSL encryption, SASL/PLAIN authentication
- 2WAY/SSL: SSL encryption, SSL authentication
- KERBEROS: no SSL encryption, Kerberos GSSAPI authentication
- SSL/KERBEROS SSL encryption, Kerberos GSSAPI authentication
- LDAP Authentication with SASL/PLAIN no SSL encryption, SASL/PLAIN authentication using LDAP
- LDAP Authorizer with SASL/PLAIN no SSL encryption, SASL/PLAIN authentication, LDAP Authorizer for ACL authorization
- RBAC with SASL/PLAIN RBAC with no SSL encryption, SASL/PLAIN authentication
Using Multi-Data-Center setup with πΊπΈ and πͺπΊ clusters
- PLAINTEXT: no security
- SASL/PLAIN: no SSL encryption, SASL/PLAIN authentication
- KERBEROS: no SSL encryption, Kerberos GSSAPI authentication
- π» Control Center
- πΎ Tiered Storage
- Tiered storage with AWS S3
- Tiered storage with Minio (unsupported)
- β Confluent Rebalancer
- π΄ JMS Client
- ποΈ Audit Logs
- ποΈ Confluent Schema Registry Security Plugin
- ποΈ Confluent REST Proxy Security Plugin with SASL/SSL and 2WAY/SSL Principal Propagation
- β»οΈ Cluster Linking
- π Testing RBAC with Azure AD
- π Schema Validation on Confluent Server Schema Validation on Confluent Server
- π Secrets Management with Connect
Easily play with Confluent Platform Ansible playbooks by using Ubuntu based Docker images generated daily from this cp-ansible-playground repository
There is also a Confluent Cloud version available here
- π¬ Separate principals when using connectors: example with SFTP source
- π How to write logs to files when using docker-compose
- π’ Publish logs to kafka with Elastic Filebeat
- π .NET basic producer
- π KafkaJS client (producer/consumer)
- πΆ Monitor Confluent Platform with Datadog
- π» Testing KIP-108 Create Topic Policy
- π Monitoring Demo
- π΅οΈββοΈ Kafka Connect Sink Monitoring Demo Showcase different Kafka Connect Sink troubleshooting scenarios
- π« Multiple Event Types in the Same Topic