databricks-industry-solutions/product-search

Creating a single-node job cluster fails on Azure

Opened this issue · 0 comments

The RUNME notebook fails in Azure because of

Unexpected user error while preparing the cluster for the job. Cause: INVALID_PARAMETER_VALUE: Spark Conf: spark.databricks.cluster.profile is not allowed when choosing an access mode

I am not setting access mode explicitly in the job API json input. I tried explicitly setting it and it didn't work either. This issue shall remain open until the cloud discrepancy issue with resolved, or if I create a workaround in the RUNME notebook.

Because of this, the integration test for Azure is temporarily disabled.