Export databases but no tables
Closed this issue · 1 comments
Is possible to export just the database names in a workspace without the table definitions? I am asking because some databases/tables point to Data Lake Storage Gen2 the cluster and I get an error when trying to export a table that is located on the Data Lake Storage Gen2 even though Pass Through Credentials are enables.
The error I get is
ERROR:
com.databricks.backend.daemon.data.client.adl.AzureCredentialNotFoundException: Could not find ADLS Gen2 Token
Alternatively, is there a way to query the databases that exist in advance in order to pick which databases to export. I checked the Databricks CLI but I don't think it has a metadata (metastore) list function.
Basically, I only care about tables that are stored in DBFS but not on the Azure Data Lake Storage
Although it throws an error, it should still be exporting the rest of the tables.
If you need a list of databases, you can pull that list through a notebook first by running spark.sql('show databases').collect()
If you need to find the database, I'd suggest using the notebook first to identify which database you need.