Trino attempts to list views from an iceberg REST catalog that does not support views
Closed this issue · 1 comments
jhatcher1 commented
Trino version: 455
We have Trino configured with an iceberg REST catalog that does not support views, and does not implement the view api endpoints. From the iceberg open API spec :
/v1/config List all catalog configuration settings
The catalog configuration also holds an optional endpoints field that contains information about the endpoints supported by the server. If a
server does not send the endpoints field, a default set of endpoints is assumed:
GET /v1/{prefix}/namespaces
POST /v1/{prefix}/namespaces
GET /v1/{prefix}/namespaces/{namespace}
DELETE /v1/{prefix}/namespaces/{namespace}
POST /v1/{prefix}/namespaces/{namespace}/properties
GET /v1/{prefix}/namespaces/{namespace}/tables
POST /v1/{prefix}/namespaces/{namespace}/tables
GET /v1/{prefix}/namespaces/{namespace}/tables/{table}
POST /v1/{prefix}/namespaces/{namespace}/tables/{table}
DELETE /v1/{prefix}/namespaces/{namespace}/tables/{table}
POST /v1/{prefix}/namespaces/{namespace}/register
POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/metrics
POST /v1/{prefix}/tables/rename
POST /v1/{prefix}/transactions/commit
Our REST catalog does not return the endpoints field, so the above list of endpoints are the set of endpoints that should be assumed.
curl localhost:8080/api/v1/config
{"defaults":{},"overrides":{"warehouse":null}}
When I run the following query in Trino, I get a stack trace that indicates Trino is trying to call some of the view endpoints.
SHOW TABLES IN iceberg.my_schema;
Table
-------------------------------------------------
my_table
(1 row)
Query 20241216_163128_00003_im768, FINISHED, 1 node
Splits: 19 total, 19 done (100.00%)
1.23 [1 rows, 85B] [0 rows/s, 69B/s]
2024-12-16T16:31:29.689Z WARN SplitRunner-20241216_163128_00003_im768.2.0.0-1-222 org.apache.iceberg.rest.ErrorHandlers Unable to parse error response
java.lang.IllegalArgumentException: Invalid response, missing field: code
at org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:145)
at org.apache.iceberg.rest.responses.ErrorResponse$Builder.build(ErrorResponse.java:130)
at org.apache.iceberg.rest.responses.ErrorResponseParser.fromJson(ErrorResponseParser.java:90)
at org.apache.iceberg.util.JsonUtil.parse(JsonUtil.java:99)
at org.apache.iceberg.rest.responses.ErrorResponseParser.fromJson(ErrorResponseParser.java:71)
at org.apache.iceberg.rest.ErrorHandlers$DefaultErrorHandler.parseResponse(ErrorHandlers.java:194)
at org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:181)
at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:323)
at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:262)
at org.apache.iceberg.rest.HTTPClient.get(HTTPClient.java:358)
at org.apache.iceberg.rest.RESTClient.get(RESTClient.java:96)
at org.apache.iceberg.rest.RESTSessionCatalog.listViews(RESTSessionCatalog.java:1133)
at io.trino.plugin.iceberg.catalog.rest.TrinoRestCatalog.lambda$listTables$3(TrinoRestCatalog.java:215)
at io.trino.plugin.iceberg.catalog.rest.TrinoRestCatalog.listTableIdentifiers(TrinoRestCatalog.java:240)
at io.trino.plugin.iceberg.catalog.rest.TrinoRestCatalog.listTables(TrinoRestCatalog.java:215)
at io.trino.plugin.iceberg.IcebergMetadata.listTables(IcebergMetadata.java:707)
at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.listTables(ClassLoaderSafeConnectorMetadata.java:287)
at io.trino.tracing.TracingConnectorMetadata.listTables(TracingConnectorMetadata.java:258)
at io.trino.metadata.MetadataManager.listTables(MetadataManager.java:542)
at io.trino.tracing.TracingMetadata.listTables(TracingMetadata.java:320)
at io.trino.metadata.MetadataListing.doListTables(MetadataListing.java:128)
at io.trino.metadata.MetadataListing.listTables(MetadataListing.java:119)
at io.trino.connector.informationschema.InformationSchemaPageSource.addTablesRecords(InformationSchemaPageSource.java:292)
at io.trino.connector.informationschema.InformationSchemaPageSource.buildPages(InformationSchemaPageSource.java:221)
at io.trino.connector.informationschema.InformationSchemaPageSource.getNextPage(InformationSchemaPageSource.java:185)
at io.trino.operator.ScanFilterAndProjectOperator$ConnectorPageSourceToPages.process(ScanFilterAndProjectOperator.java:381)
at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:261)
at io.trino.operator.WorkProcessorUtils$YieldingProcess.process(WorkProcessorUtils.java:181)
at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:346)
at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:346)
at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:346)
at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:261)
at io.trino.operator.WorkProcessorUtils$BlockingProcess.process(WorkProcessorUtils.java:207)
at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
at io.trino.operator.WorkProcessorUtils.lambda$flatten$6(WorkProcessorUtils.java:317)
at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:359)
at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
at io.trino.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:346)
at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:261)
at io.trino.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:240)
at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
at io.trino.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:261)
at io.trino.operator.WorkProcessorUtils.lambda$finishWhen$3(WorkProcessorUtils.java:255)
at io.trino.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:423)
at io.trino.operator.WorkProcessorSourceOperatorAdapter.getOutput(WorkProcessorSourceOperatorAdapter.java:133)
at io.trino.operator.Driver.processInternal(Driver.java:403)
at io.trino.operator.Driver.lambda$process$8(Driver.java:306)
at io.trino.operator.Driver.tryWithLock(Driver.java:709)
at io.trino.operator.Driver.process(Driver.java:298)
at io.trino.operator.Driver.processForDuration(Driver.java:269)
at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:890)
at io.trino.execution.executor.dedicated.SplitProcessor.run(SplitProcessor.java:77)
at io.trino.execution.executor.dedicated.TaskEntry$VersionEmbedderBridge.lambda$run$0(TaskEntry.java:201)
at io.trino.$gen.Trino_455____20241216_162942_2.run(Unknown Source)
at io.trino.execution.executor.dedicated.TaskEntry$VersionEmbedderBridge.run(TaskEntry.java:202)
at io.trino.execution.executor.scheduler.FairScheduler.runTask(FairScheduler.java:172)
at io.trino.execution.executor.scheduler.FairScheduler.lambda$submit$0(FairScheduler.java:159)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:76)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1570)
Expected behaviour:
If the REST catalog doesn't return supported endpoints from v1/config, Trino should assume the default list of endpoints, and not attempt to call the view endpoints.
ebyhr commented
I think the behavior is fixed in the latest version. Please upgrade your cluster.