/cloud-concierge

Terraform best practices as a pull request. cloud-concierge creates a Pull Request in your repo to codify resources outside of Terraform Control, detect drift, estimate cloud costs, identify security risks, and more.

Primary LanguageGoApache License 2.0Apache-2.0

cloud-concierge

Motivation

Many teams build their own Terraform management "stacks" using major cloud provider state backends and tools like Atlantis for running plan and apply and state-locking.

For more sophisticated tooling, some may turn to tools like Terraform Cloud, Scalr, Spacelift and Firefly. We find, however, that these tool's pricing can become particularly onerous when wanting to self-host runners or access the most desired features like drift detection, security scanning, etc.

Why Cloud Concierge?

cloud-concierge is a container that integrates with your existing Terraform management stack. All results and codified resources are output via a digestible Pull Request to a repository of your choice, providing you with a "State of Cloud" report in a GitOps manner. It provides:

  • ✅ Cloud codification, identify un-managed resources and generate corresponding Terraform code and import statements/import blocks

  • ✅ Drift detection

  • ✅ Flag accounts creating changes outside your Terraform workflow

  • ✅ Whole-cloud cost estimation, powered by Infracost

  • ✅ Whole-cloud security scanning, powered by tfsec (checkov integration coming soon)

Getting Started

  1. Retrieve an organization token from the dragondrop.cloud management platform here.
  2. Configure your environment variable file. This determines the execution behavior of the container. We provide example env configuration files for:

Detailed documentation on environment variables needed can be found here.

While Cloud Concierge validates environment variable formats upon start-up, we provide a UI for client-side validation of env vars within the dragondrop.cloud platform should faster iteration be desired.

  1. Run the container with the following command:
docker run --env-file ./path/to/my/env-file.env -v main:/main -w /main  dragondropcloud/cloud-concierge:latest
  1. If using Terraform >= 1.5, Cloud Concierge generates import blocks for newly codified resources directly. If using Terraform < 1.5, we generate a terraform import command for each resource. These commands can be run manually, or programmatically in a plan and apply manner using our GitHub Action.

Running on a schedule

A common use case is to want to regularly scan for drift and un-codified resources. Cloud Concierge can easily be run on a cron schedule using GitHub Actions. See our example workflow.

How does it work?

  1. cloud-concierge creates a representation of your cloud infrastructure as Terraform. Only read-only access should be given to cloud-concierge.
  2. This representation is compared against your state files to detect drift, and identify resources outside of Terraform control
  3. Static security scans and cost estimation is performed on the Terraform representation
  4. Results and code are summarized in a Pull Request within the repository of your choice

Telemetry

For OSS usage, Cloud Concierge only logs data to the dragondrop API whenever a container execution is started. This method can be viewed here.

Jobs managed by the dragondrop platform log statuses over the course of the job execution and anonymized data for cloud visualizations to the dragondrop API. These methods can be viewed here and here.

Our Roadmap

We are just getting started, and have a lot of exciting features on our roadmap. More details can be found here.

Contributing

Contributions in any form are highly encouraged. Check out our contributing guide to get started.

Using at Scale w/dragondrop.cloud

The cloud-concierge container is easy to manage in a single configuration. If you are looking to use cloud-concierge at scale, however, the dragondrop.cloud management platform allows you to:

  • Manage multiple cloud-concierge configurations through a user interface
  • Manage different cron jobs for executing each configuration at desired intervals
  • Consolidate visibility across all cloud-concierge executions into visualizations of drift, uncodified resources, cloud costs, and security risks.
  • Continue to self-host cloud-concierge instances within your cloud using serverless infrastructure.

Other Resources