terraform-databricks-examples

This repository contains the following:

Examples of implementing CI/CD pipelines to automate your Terraform deployments using Azure DevOps or Github Actions.
Multiple examples of Databricks workspace and resources deployment on Azure, AWS and GCP using Databricks Terraform provider.

There are two ways to use this repository:

Use examples as a reference for your own Terraform code: Please refer to examples folder for individual examples.
Reuse modules from this repository: Please refer to modules folder.

Repository structure

Code in the repository is organized into following folders:

modules - implementation of specific Terraform modules:
examples - specific instances that use Terraform modules.
cicd-pipelines - Detailed examples of implementing CI/CD pipelines to automate your Terraform deployments using Azure DevOps or Github Actions.

Repository content

Note
For detailed information about the examples, modules or CICD pipelines, refer to README.md file inside corresponding folder for a detailed guide on how to setup the CICD pipeline.

Examples

The folder examples contains the following Terraform implementation examples :

Cloud	Example	Description
Azure	adb-lakehouse	Lakehouse terraform blueprints
Azure	adb-with-private-link-standard	Provisioning Databricks on Azure with Private Link - Standard deployment
Azure	adb-basic	A basic example of Azure Databricks workspace deployment
Azure	adb-exfiltration-protection	A sample implementation of Data Exfiltration Protection
Azure	adb-external-hive-metastore	Example template to implement external hive metastore
Azure	adb-kafka	ADB - single node kafka template
Azure	adb-private-links	Azure Databricks Private Links
Azure	adb-private-links-general	Provisioning Databricks on Azure with Private Link and Data Exfiltration Protection
Azure	adb-splunk	ADB workspace with single VM splunk integration
Azure	adb-squid-proxy	ADB clusters with HTTP proxy
Azure	adb-teradata	ADB with single VM Teradata integration
Azure	adb-uc	ADB Unity Catalog Process
AWS	aws-workspace-basic	Provisioning AWS Databricks E2
AWS	aws-workspace-with-firewall	Provisioning AWS Databricks E2 with an AWS Firewall
AWS	aws-exfiltration-protection	An implementation of Data Exfiltration Protection on AWS
AWS	aws-workspace-with-private-link	Coming soon
AWS	aws-databricks-flat	AWS Databricks simple example
AWS	aws-databricks-modular-privatelink	Deploy multiple AWS Databricks workspaces
AWS	aws-databricks-uc	AWS UC
AWS	aws-databricks-uc-bootstrap	AWS UC
AWS	aws-remote-backend-infra	Simple example on remote backend
AWS	aws-workspace-config	Configure workspace objects
GCP	Coming soon

Modules

The folder modules contains the following Terraform modules :

Cloud	Module	Description
All	databricks-department-clusters	Terraform module that creates Databricks resources for a team
Azure	adb-lakehouse	Lakehouse terraform blueprints
Azure	adb-with-private-link-standard	Provisioning Databricks on Azure with Private Link - Standard deployment
Azure	adb-exfiltration-protection	A sample implementation of Data Exfiltration Protection
Azure	adb-with-private-links-exfiltration-protection	Provisioning Databricks on Azure with Private Link and Data Exfiltration Protection
AWS	aws-workspace-basic	Provisioning AWS Databricks E2
AWS	aws-workspace-with-firewall	Provisioning AWS Databricks E2 with an AWS Firewall
AWS	aws-exfiltration-protection	An implementation of Data Exfiltration Protection on AWS
AWS	aws-workspace-with-private-link	Coming soon
GCP	Coming soon

CICD pipelines

The folder cicd-pipelines contains the following implementation examples of pipeline: