llm-rag-chatbot has an issue on GCP as spark.conf.get("spark.databricks.clusterUsageTags.clusterWorkers") does not work
Opened this issue · 1 comments
brianmcconnell commented
Just trying to run through the RAG LLM chatbot on GCP and the OCR loading is failing because:
spark.conf.get("spark.databricks.clusterUsageTags.clusterWorkers")
returns error
[[SQL_CONF_NOT_FOUND](https://docs.gcp.databricks.com/error-messages/error-classes.html#sql_conf_not_found)] The SQL config "spark.databricks.clusterUsageTags.clusterWorkers" cannot be found. Please verify that the config exists. SQLSTATE: 42K0I
This is not a demo issue per se but I think those tags are not guaranteed to exist.
brianmcconnell commented
None urgent as I just replaced it with a hard coded number from the cluster config