/spark-utils

Utility functions for dbt projects running on Spark

Primary LanguageMakefileApache License 2.0Apache-2.0

This dbt package contains macros that:

  • can be (re)used across dbt projects running on Spark
  • define Spark-specific implementations of dispatched macros from other packages

Installation Instructions

Check dbt Hub for the latest installation instructions, or read the docs for more information on installing packages.

Compatibility

This package provides "shims" for:

  • dbt_utils, except for:
    • dbt_utils.get_relations_by_prefix_sql
    • dbt_utils.get_tables_by_pattern_sql
    • dbt_utils.get_tables_by_prefix
    • dbt_utils.get_tables_by_pattern
  • snowplow (tested on Databricks only)

Contributing

We welcome contributions to this repo! To contribute a new feature or a fix, please open a Pull Request with 1) your changes and 2) updated documentation for the README.md file.

Getting started with dbt + Spark

Code of Conduct

Everyone interacting in the dbt project's codebases, issue trackers, chat rooms, and mailing lists is expected to follow the PyPA Code of Conduct.