Support for deleting all resource groups in Azure

[istraka]

Description

We create new resource group, which name is based version and platform, i.e. eap-8.0.0.Beta-redhat-20220713-w2k19-openjdk-17-group, ... which may be different per test run. We follow suggestion that resources in the resource group share same lifecycle hence once the test run is finished resource group may be deleted.

The feature

Sometimes test execution get stuck / fails / ... and test resources may remain up costing us money. It is inconvenient to determine all resource groups that should be cleaned for separate periodical cloudwash run. I would like to have a support for option to delete all resource groups (meeting SLA minutes criteria) except for a list of predefined resource groups (containing monitoring resources, ... )

[laDok8]

I would also probably add name matching so it behaves simmilar to VMs

[istraka]

We can either introduce a breaking change and make --all to delete all resource groups or provide another parameter, i.e. --all_groups.

The feature should consider RESOURCES_SLA_MINUTES in Azure settings. However RESOURCE_GROUP and REGION would be irrelevant for the feature.

[laDok8]

RESOURCE_GROUP: ["all"] or RESOURCE_GROUP: ["*"] could be good option

[istraka]

Regex would be better since all resource group may exists.

[laDok8]

final template could look like this

AZURE:
    AUTH:
        CLIENT_ID:
        SECRET_ID:
        TENANT_ID:
        SUBSCRIPTION_ID:
        # Multiple resource groups can be added like ["foo", "bar"] or ["*"] for all RGs
        RESOURCE_GROUPS: []
        # Multiple regions can be added like ["eu-north", "us-east-2", "us-west"] or ["all"] for all regions
        REGIONS: []
    CRITERIA:
        VM:
            # The VM to be deleted with prepend string, e.g VM name that starts with 'test'
            DELETE_VM: 'test'
            # Number of minutes the deletable VM should be allowed to live, e.g 120 minutes = 2 Hours
            SLA_MINUTES: 120
        DISC:
            UNASSIGNED: True
        NIC:
            UNASSIGNED: True
        PUBLIC_IP:
            UNASSIGNED: True
        RESOURCE_GROUP:
            # Group to be deleted with prepend string, e.g resource group name that starts with 'test'
            DELETE_GROUP: 'test'
            # Delete Resource Group if all resources in resource group crossing SLA minutes
            RESOURCES_SLA_MINUTES:
    EXCEPTIONS:
        VM:
            # VM names that would be skipped from cleanup
            VM_LIST: []
            # VMs that would be stopped from current running state
            STOP_LIST: []
        GROUP:
            # Resource groups that would be skipped from cleanup
            RG_LIST: []

[laDok8]

@jyejare WDYT?

[jyejare]

Good Discussion @laDok8 and @istraka !!

I certainly agree with both of you here for this requirement here. @laDok8 The example template you have shown above is what exactly we should do.

We can iterate over RESOURCE_GROUPS and REGIONS isfall is set else just iterate over the given regions. The docstring over setting is recommended. Very similar to AWS Regions implementation.

Also, I prefer all over * because its less likely anyone name the resource group as all , if we do find that's issue we will fix it in future releases.

@laDok8 The idea of criteria for Resource group where we are willing to add name is good. But we need to add one more improvement to this criteria is a logic parameter where there are more than one criteria for any resources to delete. So that people have the choice to match all criteria or any of them.

So for Resource Group deletion it would look like:

RESOURCE_GROUP:
    # Select AND to match all criterias or OR to match any one of them
    LOGIC: AND
    # Group to be deleted with prepend string, e.g resource group name that starts with 'test'
    DELETE_GROUP: 'test'
    # Delete Resource Group if all resources in resource group crossing SLA minutes
    RESOURCES_SLA_MINUTES:

But that's not a blocker for this issues implementation if @istraka thinks the same.

Hope that answers !!

@istraka Go ahead and assign that to yourself for implementation .

[laDok8]

currently * is forbidden as RG name, but there is always option someone could name it all

Resource group names can only include alphanumeric, underscore, parentheses, hyphen, period (except at end), and Unicode characters that match the allowed characters. (taken from azure group creation page)

as for logical operands IMO the would be only adding unnecessary complexity as matching OR with more criterias doesn't seem intuitive to me and other behaviours can be configured with default AND

[jyejare]

We could have cast * to all but, I believe let's stick to the standard all we already started using.

I see that there are some requests coming to either match one or all criteria hence I am planning to add that logical operands. By default we can set it to AND, and user can choose to switch to OR.