Access kafka cluster topic in confluent cloud
Opened this issue · 2 comments
arshahmad1 commented
I'm trying to connect this connector to the kafka topic present in my confluent cloud. I was able to access the confluent kafka topic from an ec2 instance. Now I'm creating this connector in same VPC, subnet and attaching same security group as of ec2 instance but I'm unable to get the data.
These are the configurations that I'm using but I'm getting timeout
iceberg.kafka.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<CONFLUENT_API_KEY>" password="<CONFLUENT_API_SECRET>";
iceberg.kafka.security.protocol=SASL_SSL
iceberg.kafka.sasl.mechanism=PLAIN
connector.class=io.tabular.iceberg.connect.IcebergSinkConnector
table.write-format=parquet
iceberg.tables.evolve-schema-enabled=true
iceberg.fs.s3a.path.style.access=true
table.namespace=pocckafkaflink2
iceberg.catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog
tasks.max=1
topics=confluent_test_topic2
iceberg.catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO
iceberg.catalog.client.region=us-east-1
iceberg.fs.s3a.aws.credentials.provider=com.amazonaws.auth.DefaultAWSCredentialsProviderChain
iceberg.fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem
iceberg.tables=pocckafkaflink2.mskmytopicmaib
value.converter.schemas.enable=false
iceberg.catalog.warehouse=s3://bucket/data4
value.converter=org.apache.kafka.connect.json.JsonConverter
table.auto-create=true
arshahmad1 commented
is this the correct format to provide sasl.jaas.config
, security.protocol
and sasl.mechanism
shamildag commented
@arshahmad1 Hi, Muhammad, have you found the fix for this problem ?