A simple node app to test GCP stack deployments.
Follow the instructions here to set up Google Workspaces.
Save the following details:
- Organization Domain: This is generally the web domain you configured your
Google Workspaces account with. Example:
mydomain.com
- Customer ID: This is the random ID associated with your GCP account. Details on how to find it here: https://support.google.com/cloudidentity/answer/10070793
The trick here is to organize projects into folders that allow us to have a highly privileged Terraform agent across all projects managed by IaC, while keeping it from being able to self-escalate its own permissions beyond its sandbox and thus throughout the entire GCP account.
# Folder / Project Structure and Service Accounts
📁 Terraform-Managed-Resources (Folder)
├── 📁 Terraform-Managed-Projects (Folder)
│ ├── 📁 My-App (Folder)
│ │ └── 🚀 My-App (Project)
│ │ ├── 👤 Resource1 (Service Account)
│ │ └── 👤 Resource2 (Service Account)
│ │
│ ├── 📁 My-App-Dev (Folder)
│ │ ├── 🚀 My-App-Staging (Project)
│ │ │ └── 👤 Resource1 (Service Account)
│ │ │
│ │ ├── 🚀 My-App-PR-456 (Project)
│ │ │ └── 👤 Resource1 (Service Account)
│ │ │
│ │ └── 🚀 My-App-PR-123 (Project)
│ │ └── 👤 Resource1 (Service Account)
│ │
│ │
│ ├── 📁 Some-Other-App (Folder)
│ │ └── 🚀 Some-Other-App (Project)
│ │ └── 👤 Resource1 (Service Account)
│ │
│ └── 📁 Some-Other-App-Dev (Folder)
│ ├── 🚀 Some-Other-App-Staging (Project)
│ │ └── 👤 Resource1 (Service Account)
│ │
│ └── 🚀 Some-Other-App-PR-678 (Project)
│ └── 👤 Resource1 (Service Account)
│
│
└── 🚀 terraform-agents
└── 👤 Terraform (Service Account)
📁 Terraform-Managed-Resources (Folder): All Terraform-managed resources including the Terraform Agent Service Account are contained within a single root folder. This not only gives us the ability to keep these resources organized, but it gives us a way to give the Terraform Agent the high level of permissions it needs in order to manage project resources while keeping it contained within the account.
📁 Terraform-Managed-Projects (Folder): Within the root IaC folder, all projects are contained within a sub-folder. This allows us to create the service accounts required to manage the resources for a given app or environment without allowing them to escalate the scope of their permissions to the level of the Terraform Agent and thus edit or affect resources for an app or environment that they should not have access to.
🚀 Terraform-Managed-Projects (Project): Each app or environment will need a project to associate the service accounts required to manage its resources. Depending on system architecture needs over time, this could represent any number of applications, each of which will likely require at the very least one development and one production environment. For some applications, we may even want to automate the launch of unique environments on pull request "open" events.
🚀 terraform-agents (Project): Every service account needs a project. This project only exists in order to create a service account for the terraform agent that is privileged enough to manage all projects managed by Terraform, while limiting it to only IaC projects.
An example of this type of architecture can be found here
Using the React app in the examples folder of this repository, you may create a build and then push the static assets in /dist to the storage bucket to deploy the frontend.
A good network module can be found here. This may be overkill for what we are trying to do but it's something to fall back on.
We can also use the default network to get started.
We will be using Cloud Run to orchestrate and manage container clusters.
Cloud Run Terraform Simple Example
Cloud Run Terraform Example with VPC
Cloud Run Simple Example Blog Post
Questions:
- What inputs does a Cloud Run module need in order to launch successfully?
Use GitHub Actions workflows to test, build and deploy code changes.
You will need a service account for the Terraform agent to interact with GCP. There are mulitple ways to perform this. Ideally, what you want to do is use Workload Identity Federation. We're going to do a classic service account with a credentials json for speed and simplicity. In the future we should replace this with WIF. It's really not that hard.
This is a dump of actions, not necessarily meant as instructions. It's more of a log of what I'm doing. It will need to be refined once everything is working and ideally repeated for verification.
Terraform Cloud:
- Create a Terraform Cloud workspace
- Create a GCP Service Account
- Create a
GCP_CREDENTIALS
variable in TF Cloud - Copy the Service Account credentials json and set the value of the
GCP_CREDENTIALS
to the contents of the json. - Copy the generated
main.tf
file from Terraform Cloud, particularly the config with the organization and workspaces values
terraform {
cloud {
organization = "cyberworld-builders"
workspaces {
name = "todo-test"
}
required_providers {
google = {
source = "hashicorp/google"
version = ">= 4.45.0"
}
}
}
}
- Tested with
terraform plan
command locally
Google Cloud:
- Created a
Terraform-Managed-Resources
Folder
NOTE: there was a blog post i had open earlier that documented this as gcloud commands. This is the ideal method because it's much easier to document and source control than console navigation instructions. It sucks that I lost it because it was dope and it looked comprehensive.
- Create a
terraform
project for shared resources.- initially created the project on the root level with no organization
- create a service account under the terraform project
- copy the json to the
NEXT STEPS:
- Create a demo project for testing.
- Add roles to the servcie account
- Set up CI for Terraform changes
(Need to determine the registry solution. Are we using the GitHub Registry or the GCP registry. Seems like GCP is easier to authenticate. GitHub is better for hosting NPM packages.)
- Create a GitHub Action for building the api image with Docker
Many times, especially in the initial setup process, resources need to be created in GCP first and then imported into IaC (Terraform). The following are some useful commands for
gcloud
andterraform
command line interfaces for launching and/or importing resources.
Making a small number of basic one-time changes is often quick and simple to perform directly within the GCP console UI. Once you find that your changes are growing in number and complexity and/or need to be performed multiple times or reproduced; it makes more sense to use the CLI.
The CLI has the following advantages in this case:
- It is easier to document, track and share precise instructions.
- The console requires your browser to use a lot of your system memory and network bandwidth. Users often find that opening up mulitple tabs causes your machine to lag and screen shares can crash.
- Navigating the console is often a more complex experience and can lead to more user errors.
Account Configuration Commands:
Additional information on setting gcloud account configs can be found here.
# List Configurations
gcloud config list
# Create Configuration
gcloud config configurations create config-name
# Set Project
gcloud config set project my-project-id
# Set Account
gcloud config set account my-account@example.com
# Activate Config (when managing mulitple accounts)
gcloud config configurations activate config-name
Create Folder and Project Structure:
# Create Root Folder
gcloud resource-manager folders create --display-name=Terraform-Managed-Resources --organization=ORGANIZATION_ID
# Create Terraform Agents Project
gcloud projects create terraform-agents --folder=IAC_ROOT_FOLDER_ID
# Create Terraform Managed Projects Folder
gcloud resource-manager folders create --display-name=Terraform-Managed-Projects --folder=IAC_ROOT_FOLDER_ID
Once certain resources are created in the console they will need to imported back into the Terraform state so that they can be tracked by IaC.