- Clone the repository
- Enter the repository directory
- Build the provider using the following
Makefile
command:
make dep
make build
- The provider binary will be available in the
bin
directory
This provider uses Go modules. Please see the Go documentation for the most up-to-date information about using Go modules.
To add a new dependency github.com/author/dependency
to your Terraform provider:
go get github.com/author/dependency
go mod tidy
Then commit the changes to go.mod
and go.sum
.
- Create an API Token to use in the provider. We recommend creating an organization API token since it is the most flexible but the type of your API token will depend on your use case.
- Create a
main.tf
file with the following content:
terraform {
required_providers {
astro = {
source = "astronomer/astro"
}
}
}
provider "astro" {
organization_id = "<cuid>"
}
# your terraform commands here
- Run the following commands to apply the provider:
export ASTRO_API_TOKEN=<token>
terraform init # only needed the first time - initializes a working directory and downloads the necessary provider plugins and modules and setting up the backend for storing your infrastructure's state
terraform plan # creates a plan consisting of a set of changes that will make your resources match your configuration
terraform apply # performs a plan just like terraform plan does, but then actually carries out the planned changes to each resource using the relevant infrastructure provider's API
If you wish to work on the provider, you'll first need Go installed on your machine (see Requirements above).
To compile the provider, see Building The Provider.
To add example docs, add the correspond .tf
files to the examples
directory. These should be added for every new data source and resource.
To run terraform with the provider, create a .terraformrc
file in your home directory (~
) with the following content to override the provider installation with the local build:
provider_installation {
dev_overrides {
"registry.terraform.io/astronomer/astro" = "~/terraform-provider-astro/bin" # Your path to the provider binary
}
direct {}
}
terraform {
required_providers {
astro = {
source = "astronomer/astro"
}
}
}
# provider configuration
provider "astro" {
organization_id = "<cuid>"
}
# get information on an existing workspace
data "astro_workspace" "example" {
id = "<cuid>"
}
# output the workspace data to the terminal
output "data_workspace_example" {
value = data.astro_workspace.example
}
# create a new workspace
resource "astro_workspace" "tf_workspace" {
name = "my workspace"
description = "my first workspace"
cicd_enforced_default = false
}
# output the newly created workspace resource to the terminal
output "terraform_workspace" {
value = astro_workspace.tf_workspace
}
# create a new cluster resource
resource "astro_cluster" "tf_cluster" {
type = "DEDICATED"
name = "my first cluster"
region = "us-east-1"
cloud_provider = "AWS"
vpc_subnet_range = "172.20.0.0/20"
workspace_ids = [astro_workspace.tf_workspace.id, data.astro_workspace.example.id]
timeouts = {
create = "3h"
update = "2h"
delete = "20m"
}
}
# create a new dedicated deployment resource in that cluster
resource "astro_deployment" "tf_dedicated_deployment" {
name = "my first dedicated deployment"
description = ""
cluster_id = astro_cluster.tf_cluster.id
type = "DEDICATED"
contact_emails = ["example@astronomer.io"]
default_task_pod_cpu = "0.25"
default_task_pod_memory = "0.5Gi"
executor = "KUBERNETES"
is_cicd_enforced = true
is_dag_deploy_enabled = true
is_development_mode = false
is_high_availability = true
resource_quota_cpu = "10"
resource_quota_memory = "20Gi"
scheduler_size = "SMALL"
workspace_id = astro_workspace.tf_workspace.id
environment_variables = [{
key = "key1"
value = "value1"
is_secret = false
}]
}
# create a new standard deployment resource
resource "astro_deployment" "tf_standard_deployment" {
name = "my first standard deployment"
description = ""
type = "STANDARD"
cloud_provider = "AWS"
region = "us-east-1"
contact_emails = []
default_task_pod_cpu = "0.25"
default_task_pod_memory = "0.5Gi"
executor = "CELERY"
is_cicd_enforced = true
is_dag_deploy_enabled = true
is_development_mode = false
is_high_availability = false
resource_quota_cpu = "10"
resource_quota_memory = "20Gi"
scheduler_size = "SMALL"
workspace_id = astro_workspace.tf_workspace.id
environment_variables = []
worker_queues = [{
name = "default"
is_default = true
astro_machine = "A5"
max_worker_count = 10
min_worker_count = 0
worker_concurrency = 1
}]
}
Unit tests can be run with make test
.
Acceptance integration tests use a Terraform CLI binary to run real Terraform commands against the Astro API. The goal is to approximate using the provider with Terraform in production as closely as possible.
Using the terraform-plugin-testing framework, each resource.Test
runs an acceptance test on a resource.
ProtoV6ProviderFactories
: map of the provider factories that the test suite will use to create the provider - just has theastronomer
providerPreCheck
: a function that runs before the test suite starts to check that all the required environment variables are setSteps
: a list ofterraform apply
sequences that the test suite will run. Each step is aresource.TestStep
that contains aConfig
andCheck
function.Config
: the Terraform configuration that the test will run (ie. the.tf
file)Check
: function that will verify the state of the resources after theterraform apply
command has run.
In order to run the full suite of Acceptance tests, run make testacc
.
You will also need to set all the environment variables described in internal/provider/provider_test_utils.go
.
The acceptance tests will run against the Astronomer API and create/read/update/delete real resources.
The Astro Terraform Import Script is a tool designed to help you import existing Astro resources into your Terraform configuration. Currently, this script automates the process of generating Terraform import blocks and resource configurations for the following resources: workspaces, deployments, clusters, hybrid cluster workspace authorizations, API tokens, teams, team roles, and user roles.
To use the import script, download the terraform-provider-astro-import-script
executable file from releases based on your OS and architecture and run it with the following command:
On Unix-based systems:
chmod +x terraform-provider-astro-import-script_<version-number>_<os>_<arc>
./terraform-provider-astro-import-script_<version-number>_<os>_<arc> [options]
On Windows:
.\terraform-provider-astro-import-script_<version-number>_<os>_<arc>.exe [options]
-resources
: Comma-separated list of resources to import. Accepted values are workspace, deployment, cluster, api_token, team, team_roles, user_roles.-token
: API token to authenticate with the Astro platform. If not provided, the script will attempt to use theASTRO_API_TOKEN
environment variable.-organizationId
: Organization ID to import resources from.-runTerraformInit
: Runterraform init
after generating the import configuration. Used for initializing the Terraform state in our GitHub Actions.-help
: Display help information.
-
Import workspaces and deployments:
./terraform-provider-astro-import-script_<version-number>_<os>_<arc> -resources workspace,deployment -token your_api_token -organizationId your_org_id
-
Import all supported resources and run Terraform init:
./terraform-provider-astro-import-script_<version-number>_<os>_<arc> -resources workspace,deployment,cluster,api_token,team,team_roles,user_roles -token your_api_token -organizationId your_org_id -runTerraformInit
-
Use a different API host (e.g., dev environment):
./terraform-provider-astro-import-script_<version-number>_<os>_<arc> -resources workspace -token your_api_token -organizationId your_org_id
The script will generate two main files:
import.tf
: Contains the Terraform import blocks for the specified resources.generated.tf
: Contains the Terraform resource configurations for the imported resources.
- Ensure you have the necessary permissions in your Astro organization to access the resources you're attempting to import.
- The generated Terraform configurations may require some manual adjustment to match your specific requirements or to resolve any conflicts.
- Always review the generated files before applying them to your Terraform state.
-
What resources can I manage with this Terraform provider?
- Workspaces, deployments, clusters, hybrid cluster workspace authorizations, API tokens, teams, team roles, and user roles.
-
How do I authenticate with the Astro API?
- Use an API token set as the
ASTRO_API_TOKEN
environment variable or add it to the provider configuration.
- Use an API token set as the
-
Can I import existing Astro resources into Terraform?
- Yes, use the Astro Terraform Import Script to generate import blocks and resource configurations.
-
What Terraform versions are required?
- Terraform >= 1.7.
-
How can I contribute to the provider's development?
- Submit pull requests, report issues, or suggest improvements on the GitHub repository.
-
Issue: 401 Unauthorized error when running
terraform plan
orterraform apply
Solution: Your API token may have expired. Update your
ASTRO_API_TOKEN
environment variable with a fresh token:export ASTRO_API_TOKEN=<your-new-token>
-
Issue: Import script fails to find resources
Solution:
- Ensure you have the correct permissions in your Astro organization.
- Verify that your API token is valid and has the necessary scopes and permissions.
- Double-check the organization ID provided to the script.
-
Issue: "Error: Invalid provider configuration" when initializing Terraform
Solution: Ensure your
.terraformrc
file is correctly set up, especially if you're using a local build of the provider for development.
If you encounter any issues not listed here, please check the GitHub Issues page or open a new issue with details about your problem.