Azure/spark-cdm-connector

Error while connecting to model.json using spark pools on azure

Closed this issue · 1 comments

I am trying to connect to a model.json file located in a data lake folder.
Below is the code that I am using to connect.

val df = spark.read.format("com.microsoft.cdm")
.option("storage",".dfs.core.windows.net")
.option("cdmModel", "https://....../model.json")
.option("entity", "account")
.load()

After running the cell with the above code in it, the following is the error.

Screen Shot 2022-11-30 at 3 32 11 pm

Any help would be appreciated.

You should check the documents. Error message says you are missing the "manifestPath" argument.
https://github.com/Azure/spark-cdm-connector/blob/master/documentation/overview.md#common-read-and-write-options

The samples even show the correct usage.
https://github.com/Azure/spark-cdm-connector/blob/master/samples/SparkCDMsample.scala