[BUG]pulsar-flink catalog options
bingfeng2004 opened this issue · 1 comments
Describe the bug
1.I used catalog to configure SQL-client and started SQL-client with a message indicating that the format parameter is not supported.
`catalogs:
- name: pulsarCatalog
type: pulsar-catalog
default-database: default
catalog-service-url: "pulsar://192.168.1.113:6650"
catalog-admin-url: "http://192.168.1.113:8080"
format: json
`
[root@node07 bin]# ./sql-client.sh embedded
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/flink-1.13.6/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
No default environment specified.
Searching for '/opt/flink-1.13.6/conf/sql-client-defaults.yaml'...found.
Reading default environment from: file:/opt/flink-1.13.6/conf/sql-client-defaults.yaml
Exception in thread "main" org.apache.flink.table.client.SqlClientException: Unexpected exception. This is a bug. Please consider filing an issue.
at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:201)
at org.apache.flink.table.client.SqlClient.main(SqlClient.java:161)
Caused by: org.apache.flink.table.api.ValidationException: Unable to create catalog 'pulsarCatalog'.
Catalog options are:
'catalog-admin-url'='http://192.168.1.113:8080'
'catalog-service-url'='pulsar://192.168.1.113:6650'
'default-database'='default'
'format'='json'
'type'='pulsar-catalog'
at org.apache.flink.table.factories.FactoryUtil.createCatalog(FactoryUtil.java:270)
at org.apache.flink.table.client.gateway.context.LegacyTableEnvironmentInitializer.createCatalog(LegacyTableEnvironmentInitializer.java:217)
at org.apache.flink.table.client.gateway.context.LegacyTableEnvironmentInitializer.lambda$initializeCatalogs$1(LegacyTableEnvironmentInitializer.java:120)
at java.util.HashMap.forEach(HashMap.java:1288)
at org.apache.flink.table.client.gateway.context.LegacyTableEnvironmentInitializer.initializeCatalogs(LegacyTableEnvironmentInitializer.java:117)
at org.apache.flink.table.client.gateway.context.LegacyTableEnvironmentInitializer.initializeSessionState(LegacyTableEnvironmentInitializer.java:105)
at org.apache.flink.table.client.gateway.context.SessionContext.create(SessionContext.java:233)
at org.apache.flink.table.client.gateway.local.LocalContextUtils.buildSessionContext(LocalContextUtils.java:100)
at org.apache.flink.table.client.gateway.local.LocalExecutor.openSession(LocalExecutor.java:91)
at org.apache.flink.table.client.SqlClient.start(SqlClient.java:88)
at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:187)
... 1 more
Caused by: org.apache.flink.table.api.ValidationException: Unsupported options found for 'pulsar-catalog'.
**Unsupported options:
format**
Supported options:
catalog-admin-url
catalog-auth-params
catalog-auth-plugin
catalog-service-url
catalog-tenant
default-database
property-version
pulsar-version
table-default-partitions
at org.apache.flink.table.factories.FactoryUtil.validateUnconsumedKeys(FactoryUtil.java:398)
at org.apache.flink.table.factories.FactoryUtil$CatalogFactoryHelper.validate(FactoryUtil.java:563)
at org.apache.flink.streaming.connectors.pulsar.catalog.PulsarCatalogFactory.createCatalog(PulsarCatalogFactory.java:52)
at org.apache.flink.table.factories.FactoryUtil.createCatalog(FactoryUtil.java:267)
... 11 more
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
The Readme is described in the catalog configuration is not correct, do not agree with the parameter name in the "PulsarCatalogFactoryOptions“ class requirement
catalogs:
- name: pulsarcatalog
type: pulsar
default-database: tn/ns
service-url: "pulsar://localhost:6650"
admin-url: "http://localhost:8080"
format: json
We reimplemented the catalog for 1.13, 1.14 and master branch.
You will use the following option keys to reconfigure your catalog.
catalog-admin-url
catalog-auth-params
catalog-auth-plugin
catalog-service-url
catalog-tenant
default-database
property-version
pulsar-version
table-default-partitions