/python_app_to_databricks

CLI which aides in the create of a notebook application from a source application.

Primary LanguagePythonMIT LicenseMIT

Generate Notebook Application

This python CLI generates a Databricks archive from a source application so that it can run in Databricks without needing to install a whl file on a cluster. This is useful for deploying applications to Databricks that are not part of the Databricks ecosystem, or for deploying applications that are not part of the Databricks ecosystem. It is also for users that do not have the ability to install packages on a Databricks cluster.

CI/CD

  • CLI Tests - The CLI is tested to ensure it works as expected.

Installation

Install dependencies with poetry:

poetry install

Testing

Beyond dependencies and a sample application nothing further is needed.

poetry run pytest

Usage

Build the databricks archive file

py2databricks build --entrypoint <entrypoint_module> --output_name databricks_archive.zip

Print the dependency tree for the application

py2databricks print_dependency_tree --entrypoint <entrypoint_module>