GCP Resource Reaper

The GCP Resource Reaper is a system to register and clean up leaked GCP resources. It is configured dynamically via protocol buffers, and runs on a given schedule until it is stopped by the user.

NOTE: This is not an officially supported Google product

Usage

Ideal use case for applications that created many GCP resources during end-to-end test. The reaper can act as a failsafe, so if resources are failed to be deleted as a result of test cancelation, test failure or a coding error, the engineer doesn't have to spend hours debugging.

Show Instructions

Download required tools
- Go
- Bazel

Clone the repository and cd into it

$ git clone https://github.com/googleinterns/cloudai-gcp-test-resource-reaper
$ cd cloudai-gcp-test-resource-reaper

Start the gRPC server
```
 $ bazel run //cmd/start_server:start_server -- -logs-project=YOUR_GCP_PROJECT -logs-name=YOUR_LOGS_NAME
```
Note: You can view logs in either the logs.txt file that was created, or in stackdriver cloud logs based off of the information you passed here.

Start the reaper manager

$ bazel run //cmd/reaper:reaper -- start

Create a new reaper

$ bazel run //cmd/reaper:reaper -- create

Follow the prompt to create a new reaper based off of which resource you want to monitor
Once the reaper is created, you can run any of the following:
- View all running reapers:
```
$ bazel run //cmd/reaper:reaper -- list
```
- View report of reaper manager of currently monitored resources and previously deleted resources
```
$ bazel run //cmd/reaper:reaper -- report
```

Delete the reaper once you are ready

$ bazel run //cmd/reaper:reaper -- delete -uuid=REAPER_UUID

Shutdown the reaper manager

$ bazel run //cmd/reaper:reaper -- shutdown

How It Works

The user configures the reaper manager over gRPC. The reaper manager holds multiple reapers, each of which monitors the resources defined in the ReaperConfig. The reaper will delete the WatchedResource once it's past its Time-To-Live (TTL).

Config Protos

There are two protos for configuring the reaper: ReaperConfig and ResourceConfig.

ReaperConfig: Defines properties of the reaper, and which resources will be monitored.

message ReaperConfig {
    repeated ResourceConfig resources = 1;
    string schedule = 2;
    string project_id = 3;
    string uuid = 4;
}

ResourceConfig: Describes a set of resources and their TTL.

message ResourceConfig {
    ResourceType resource_type = 1;
    string name_filter = 2;
    string skip_filter = 3;
    repeated string zones = 4;
    string ttl = 5;
}

Source Code Headers

Every file containing source code must include copyright and license information. This includes any JS/CSS files that you might be serving out to browsers. (This is to help well-intentioned people avoid accidental copying that doesn't comply with the license.)

Apache header:

Copyright 2020 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

brianmodel/cloudai-gcp-test-resource-reaper

GCP Resource Reaper

Usage

How It Works

Config Protos

Source Code Headers