aws/aws-sdk-ruby

Listing shared RDS Clusters raises an ArgumentError

danwanek-wm opened this issue · 6 comments

danwanek-wm commented

Describe the bug

I am trying to list shared DB Clusters and it is raising the following exception:

gems/ruby-3.2.2@aws/gems/aws-sdk-rds-1.204.0/lib/aws-sdk-rds/db_cluster.rb:3120:in `extract_id': missing required option :id (ArgumentError)

      when nil then raise ArgumentError, "missing required option :id"
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-rds-1.204.0/lib/aws-sdk-rds/db_cluster.rb:24:in `initialize'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-rds-1.204.0/lib/aws-sdk-rds/resource.rb:2746:in `new'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-rds-1.204.0/lib/aws-sdk-rds/resource.rb:2746:in `block (3 levels) in db_clusters'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-rds-1.204.0/lib/aws-sdk-rds/resource.rb:2745:in `each'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-rds-1.204.0/lib/aws-sdk-rds/resource.rb:2745:in `block (2 levels) in db_clusters'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-core-3.186.0/lib/aws-sdk-core/pageable_response.rb:191:in `each'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-rds-1.204.0/lib/aws-sdk-rds/resource.rb:2743:in `block in db_clusters'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-core-3.186.0/lib/aws-sdk-core/resources/collection.rb:101:in `each'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-core-3.186.0/lib/aws-sdk-core/resources/collection.rb:101:in `each'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-core-3.186.0/lib/aws-sdk-core/resources/collection.rb:101:in `block in non_empty_batches'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-core-3.186.0/lib/aws-sdk-core/resources/collection.rb:52:in `each'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-core-3.186.0/lib/aws-sdk-core/resources/collection.rb:52:in `each'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-core-3.186.0/lib/aws-sdk-core/resources/collection.rb:52:in `block in each'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-core-3.186.0/lib/aws-sdk-core/resources/collection.rb:58:in `each'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-core-3.186.0/lib/aws-sdk-core/resources/collection.rb:58:in `each'
        from /Users/myhome/.rvm/gems/ruby-3.2.2@aws/gems/aws-sdk-core-3.186.0/lib/aws-sdk-core/resources/collection.rb:58:in `each'
        from demo_rds.rb:16:in `<main>'

This is the code that is raising it:

rds = Aws::RDS::Resource.new

rds.db_clusters(include_shared: true).each do |c|
  puts "Status:    #{c.status}"

  c.snapshots.each do |s|
    puts "  Snapshot: #{s.snapshot_id}"
    puts "  Status:   #{s.status}"
  end
end

Expected Behavior

I expected it to print out a list of DB Clusters, including the ones shared with my Account.

Current Behavior

The backtrace is already included, but I also noticed from the AWS CLI output, that the shared clusters do not have an ID in the listed output.

> aws rds describe-db-clusters --include-shared
{
    "DBClusters": [
        {
            "Engine": "aurora-postgresql",
            "EngineVersion": "<my version>",
            "StorageEncrypted": true,
            "DBClusterArn": "arn:aws:rds:<my-region>:<myaccount>:cluster:<my-shared-cluster>",
            "AutoMinorVersionUpgrade": false
        }
    ]
}

Reproduction Steps

See above

Possible Solution

No response

Additional Information/Context

No response

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-rds

Environment details (Version of Ruby, OS environment)

Ruby 3.2.2 in RVM on OSX

alextwoods commented

Looking at the code in resources.rb:

          page.data.db_clusters.each do |d|
            batch << DBCluster.new(
              id: d.db_cluster_identifier,
              data: d,
              client: @client
            )
          end

It looks like its trying to use the db_cluster_identifier for the id. However, looking at the response shown from the CLI - its possible this field is not always returned anymore and the service needs to update their resources definition (we do not author that file, and the resources.rb is auto generated from it).

alextwoods commented

I'd recommend not using the Resource interface and instead using the client directly:

rds = Aws::RDS::Client.new
resp = rds.describe_db_clusters(include_shared: true)
resp.db_clusters.each { |c| puts c.status }
# will need to probably call describe_db_cluster_snapshots on each one individually

Docs: describe_db_cluster_snapshots

  • 👍1
mullermp commented

How old is this cluster? Identifier should be present AFAIK.

danwanek-wm commented

Thanks for the work-around @alextwoods. That works!

@mullermp - The cluster engine version is PG 11.21, so it's an older version.

mullermp commented

Agree with not using the Resource interface. That's odd that your cluster doesn't have an identifier. Maybe new clusters will have one created? Or is there a way to edit your cluster to have an identifier?

github-actions commented

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.