terraform-databricks-examples

This repository contains multiple examples of implementing CI/CD pipelines to deploy Databricks resources using Databricks Terraform provider.

General workflow

The general workflow for examples looks as following:

Changes to code are made in a separate Git branch & when changes are ready, a pull request is opened
Upon opening of the pull request, the build pipeline is triggered, and following operations are performed:
- Initializes Terraform using a remote backend to store a Terraform state.
- Perform check of the Terraform code for formatting consistency.
- Performs check of the Terraform code using terraform validate.
- Executes terraform plan to get the list changes that will be made during deployment.
If the build pipeline is executed without errors, results of terraform plan and the code could be reviewed by reviewer, and merged into the main branch.
When code is merged into the main branch, the release pipeline is triggered, and after a manual approval, changes are applied to the deployment using the terraform apply command.

Code in the repository is organized into following folders:

modules - implementation of specific Terraform modules:
- databricks-department-clusters - Terraform module that creates Databricks resources for a team.
examples - specific instances that use Terraform modules, providing CI/CD capabilities for deployment. Refer to README.md files inside specific folder:
- manual-approve-with-azure-devops - implementation of databricks-department-clusters module using Azure DevOps.
- manual-approve-with-github-actions - implementation of databricks-department-clusters module using GitHub Actions.