Azure/spark-cdm-connector

Can't read lower level manifest even though useSubManifests is set to true

Opened this issue · 0 comments

Did you read the pinned issues and search the error message?

No, but I will read and search it now before creating an issue.

Summary of issue

I followed the instructions to use the option useSubManifests = true to load an entity from the cdm folder. I used a top-level manifest at the tables level or folder and was hoping that it would read all of the sub-manifest until it found the entity. However, that wasn't the behavior. I got an error saying that it couldn't find the entity. Am I using the connector correctly or is this a bug?

Error stack trace

No response

Platform name

Databricks

Spark version

3.1.2

CDM jar version

3.1.19.5

What is the format of the data you are trying to read/write?

.csv