databrickslabs/dbx

Support for Databricks CLI versions 0.205 and above

mrmasterplan opened this issue · 1 comments

mrmasterplan commented

Expected Behavior

I expect the dbx tool to continue to support the lates and recommended versions of databricks tooling.
Databricks now recommends all users to migrate to the new Databricks CLI version 0.205 or above. (Link: https://learn.microsoft.com/en-us/azure/databricks/archive/dev-tools/cli/)
The version 0.18 is considered deprecated. DBX currently requires databricks-cli version 0.17. (See link to repo here: https://github.com/databrickslabs/dbx/blob/main/setup.py#L14)

Can we have a discussion how we continue to support the latests tool versions?

Current Behavior

DBX is incompatible with Databricks CLI version 0.205 or above.

Context

see above

Your Environment

  • dbx version used: 0.8.18
  • Databricks Runtime version: 13.3LTS
mrmasterplan commented

I see two ways to proceed:

Option 1: Use the databricks-sdk package which is available on pypi and depend on that instead of on the CLI tool.

Option 2: Depend on Databricks CLI version 0.205 and above which needs to be installed independently. It is not avialable on Pypi.

What do people prefer? I feel that option 2 is probably easier to code.