add support for missing RDS limits

Question

add support for missing RDS limits

sebasrp opened this issue 4 years ago · 5 comments

sebasrp commented 4 years ago

Feature Request

Feature Description

We are missing support for the following quotas:

ManualClusterSnapshots
DBInstanceRoles
CustomEndpointsPerDBCluster
DBClusterRoles

Use Cases

I want to get access to those limits and see my usage.

Actual Output

This was detected when collecting usage and limits for RDS

awslimitchecker --region ap-southeast-1 -P staging --service RDS --skip-ta -u -vv
awslimitchecker 9.0.0@9.0.0 is AGPL-licensed free software; all users have a right to the full source code of this version. See <https://github.com/jantman/awslimitchecker>
...
2020-09-30 17:50:42,890 [INFO rds.py:298 - awslimitchecker.services.rds._update_limits_from_api() ] Querying RDS DescribeAccountAttributes for limits
2020-09-30 17:50:42,917 [DEBUG connectionpool.py:433 - urllib3.connectionpool._make_request() ] https://rds.ap-southeast-1.amazonaws.com:443 "POST / HTTP/1.1" 200 3083
2020-09-30 17:50:42,919 [INFO rds.py:302 - awslimitchecker.services.rds._update_limits_from_api() ] RDS DescribeAccountAttributes returned unknownlimit: ManualClusterSnapshots (max: 100; used: 0)
2020-09-30 17:50:42,919 [INFO rds.py:302 - awslimitchecker.services.rds._update_limits_from_api() ] RDS DescribeAccountAttributes returned unknownlimit: DBInstanceRoles (max: 5; used: 0)
2020-09-30 17:50:42,919 [INFO rds.py:302 - awslimitchecker.services.rds._update_limits_from_api() ] RDS DescribeAccountAttributes returned unknownlimit: CustomEndpointsPerDBCluster (max: 5; used: 0)
2020-09-30 17:50:42,919 [INFO rds.py:302 - awslimitchecker.services.rds._update_limits_from_api() ] RDS DescribeAccountAttributes returned unknownlimit: DBClusterRoles (max: 5; used: 0)
2020-09-30 17:50:42,919 [DEBUG rds.py:310 - awslimitchecker.services.rds._update_limits_from_api() ] Done setting limits from API.
2020-09-30 17:50:42,919 [DEBUG rds.py:90 - awslimitchecker.services.rds.find_usage() ] Done checking usage.
2020-09-30 17:50:42,920 [INFO rds.py:298 - awslimitchecker.services.rds._update_limits_from_api() ] Querying RDS DescribeAccountAttributes for limits
2020-09-30 17:50:42,946 [DEBUG connectionpool.py:433 - urllib3.connectionpool._make_request() ] https://rds.ap-southeast-1.amazonaws.com:443 "POST / HTTP/1.1" 200 3083
2020-09-30 17:50:42,948 [INFO rds.py:302 - awslimitchecker.services.rds._update_limits_from_api() ] RDS DescribeAccountAttributes returned unknownlimit: ManualClusterSnapshots (max: 100; used: 0)
2020-09-30 17:50:42,948 [INFO rds.py:302 - awslimitchecker.services.rds._update_limits_from_api() ] RDS DescribeAccountAttributes returned unknownlimit: DBInstanceRoles (max: 5; used: 0)
2020-09-30 17:50:42,948 [INFO rds.py:302 - awslimitchecker.services.rds._update_limits_from_api() ] RDS DescribeAccountAttributes returned unknownlimit: CustomEndpointsPerDBCluster (max: 5; used: 0)
2020-09-30 17:50:42,948 [INFO rds.py:302 - awslimitchecker.services.rds._update_limits_from_api() ] RDS DescribeAccountAttributes returned unknownlimit: DBClusterRoles (max: 5; used: 0)
2020-09-30 17:50:42,948 [DEBUG rds.py:310 - awslimitchecker.services.rds._update_limits_from_api() ] Done setting limits from API.
2020-09-30 17:50:42,948 [DEBUG base.py:296 - awslimitchecker.services.base._update_service_quotas() ] Updating service quotas for RDS
RDS/DB Cluster Parameter Groups   0
RDS/DB Clusters                   0
RDS/DB instances                  0
RDS/DB parameter groups           0
RDS/DB security groups            0
RDS/DB snapshots per user         0
RDS/Event Subscriptions           0
RDS/Max auths per security group  default=0
RDS/Option Groups                 0
RDS/Read replicas per master      0
RDS/Reserved Instances            0
RDS/Storage quota (GB)            0
RDS/Subnet Groups                 0
RDS/Subnets per Subnet Group      0
RDS/VPC Security Groups           0

Testing Assistance

Yes, i will be able to assist in testing pre-release code for the feature.

Answer 1 · 2020-09-30T10:38:28.000Z

Of interest:
These quotas are present / described in RDS API_AccountQuota - but not in the RDS Quota page.

The DescribeAccountAttributes does include the expected usage and limit in the response:

<DescribeAccountAttributesResponse xmlns="http://rds.amazonaws.com/doc/2014-10-31/">
  <DescribeAccountAttributesResult>
    <AccountQuotas>
      <AccountQuota>
        <AccountQuotaName>DBInstances</AccountQuotaName>
        <Max>40</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>ReservedDBInstances</AccountQuotaName>
        <Max>40</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>AllocatedStorage</AccountQuotaName>
        <Max>100000</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>DBSecurityGroups</AccountQuotaName>
        <Max>25</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>AuthorizationsPerDBSecurityGroup</AccountQuotaName>
        <Max>20</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>DBParameterGroups</AccountQuotaName>
        <Max>50</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>ManualSnapshots</AccountQuotaName>
        <Max>100</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>ManualClusterSnapshots</AccountQuotaName>
        <Max>100</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>EventSubscriptions</AccountQuotaName>
        <Max>20</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>DBSubnetGroups</AccountQuotaName>
        <Max>50</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>OptionGroups</AccountQuotaName>
        <Max>20</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>SubnetsPerDBSubnetGroup</AccountQuotaName>
        <Max>20</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>ReadReplicasPerMaster</AccountQuotaName>
        <Max>5</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>DBInstanceRoles</AccountQuotaName>
        <Max>5</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>DBClusters</AccountQuotaName>
        <Max>40</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>DBClusterParameterGroups</AccountQuotaName>
        <Max>50</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>CustomEndpointsPerDBCluster</AccountQuotaName>
        <Max>5</Max>
        <Used>0</Used>
      </AccountQuota>
      <AccountQuota>
        <AccountQuotaName>DBClusterRoles</AccountQuotaName>
        <Max>5</Max>
        <Used>0</Used>
      </AccountQuota>
    </AccountQuotas>
  </DescribeAccountAttributesResult>
  <ResponseMetadata>
    <RequestId>1234</RequestId>
  </ResponseMetadata>
</DescribeAccountAttributesResponse>

Answer 2 · 2020-09-30T10:38:38.000Z

I am preparing a PR for this

Answer 3 · 2020-12-02T14:12:47.000Z

The PR for this has been merged, and is greatly appreciated! Apologies for letting this sit so long.

Answer 4 · 2020-12-02T14:13:21.000Z

A fix for this issue has been merged to the develop branch and is slated for release in 10.0.0 on Monday, December 7, 2020. Anyone who is able to test the fix is strongly encouraged to do so, and comment on this issue with their results. Instructions for installing awslimitchecker from a git branch are available in the Installing for Development documentation.

Answer 5 · 2020-12-07T20:50:14.000Z

This has been fixed in 10.0.0, which is now live on PyPI and is being built on the Docker hub right now. Thank you so much for the contribution!