aws-beam/aws-elixir

AWS.TimestreamWrite does not work.

Closed this issue · 5 comments

sasa-git commented

Hello.

I tried using the list_databases() function of AWS.TimestreamWrite, but I received the following response. I also tried other functions like list_tables() and list_tags_for_resource(), but they returned the same response, so there seems to be an issue with the library.

aws-elixir version: 1.0.1
elixir version: 1.16.0
os: Debian 11.7

for example:

# The credentials have been set in advence.
> credentials = :aws_credentials.get_credentials()
> client = AWS.Client.create(credentials[:access_key_id], credentials[:secret_access_key], "ap-northeast-1")

> AWS.TimestreamWrite.list_tables(client, %{})
{:error,
 {:unexpected_response,
  %{
    body: "{\"__type\":\"com.amazon.coral.service#UnknownOperationException\"}",
    headers: [
      {"x-amzn-RequestId", "04fc48ed-6dcc-41d5-bd8b-330eee0d897c"},
      {"Content-Type", "application/x-amz-json-1.0"},
      {"Content-Length", "63"},
      {"Date", "Fri, 07 Jun 2024 08:48:08 GMT"}
    ],
    status_code: 404
  }}}
> AWS.TimestreamWrite.list_tables(client, %{})
{:error,
 {:unexpected_response,
  %{
    body: "{\"__type\":\"com.amazon.coral.service#UnknownOperationException\"}",
    headers: [
      {"x-amzn-RequestId", "45966278-a641-4d75-89bb-085f6ada3367"},
      {"Content-Type", "application/x-amz-json-1.0"},
      {"Content-Length", "63"},
      {"Date", "Fri, 07 Jun 2024 09:03:16 GMT"}
    ],
    status_code: 404
  }}}

I also tried the similar AWS.TimestreamInfluxDB.list_db_instances(), and it worked without any problems, so it doesn't seem to be an authentication error.

for example:

> AWS.TimestreamInfluxDB.list_db_instances(client, %{})
{:ok, %{"items" => []},
 %{
   body: "{\"items\":[]}",
   headers: [
     {"Date", "Fri, 07 Jun 2024 09:05:31 GMT"},
     {"Content-Type", "application/x-amz-json-1.0"},
     {"Content-Length", "12"},
     {"Connection", "keep-alive"},
     {"x-amzn-RequestId", "a6b9d995-5ffb-4491-b745-b901631539d9"}
   ],
   status_code: 200
 }}

I would appreciate any solutions you could provide.
Thank you for your help :)

onno-vos-dev commented

@sasa-git I wonder if it has to do with the following block: timestream-write.json#L1459-L1461 which seems to indicate the need for a clientDiscoveredEndpoint. Something not necessary for TimestreamInfluxDB and hence why the latter probably works while the former does not 😄

What gets returned when you call the following function: (Make sure to mask anything you deem sensitive)

AWS.TimestreamWrite.describe_endpoints(client, %{})

I wonder if it may be required to link up the list_databases call with a call to the above prior to it so I'd be interested to see the response of that and we'll take it from there 👍

Edit:
If it returns a list of Endpoints unique to you which the docs seem to indicate you could use AWS.Client.put_endpoint(client, "endpoint") to set the endpoint prior to calling AWS.TimestreamWrite.list_tables(client, %{}). Ideally we should probably find a more long-term solution for this and figure out how to do this programmatically instead of leaving it up to the aws-elixir user.

sasa-git commented

@onno-vos-dev
Thank you for your response.
I executed the following code.

> AWS.TimestreamWrite.describe_endpoints(client, %{})
{:error,
 {:unexpected_response,
  %{
    body: "{\"__type\":\"com.amazon.coral.service#InvalidSignatureException\",\"message\":\"Credential should be scoped to correct service: 'timestream'. \"}",
    headers: [
      {"x-amzn-RequestId", "c331583f-8e44-4cc2-9461-a02d7df0dec8"},
      {"Content-Type", "application/x-amz-json-1.0"},
      {"Content-Length", "138"},
      {"Date", "Fri, 07 Jun 2024 11:36:22 GMT"}
    ],
    status_code: 403
  }}}
onno-vos-dev commented

@sasa-git Should be able to roll out a fix shortly 👍 This is a bug in the metadata.signing_name field 👍

sasa-git commented

@onno-vos-dev
Thank you for fixing the bug! The describe_endpoints() function is now working correctly, and using the obtained endpoint, the list_databases() function also worked properly.
Thank you for your help! 😄

> AWS.TimestreamWrite.describe_endpoints(client, %{})
{:ok,
 %{
   "Endpoints" => [
     %{
       "Address" => "[masked_endpoint]",
       "CachePeriodInMinutes" => 1440
     }
   ]
 },
 %{
   <omission>
 }}
> c2 = AWS.Client.put_endpoint(client, "[masked_endpoint]")
> AWS.TimestreamWrite.list_databases(c2, %{})
{:ok,
 %{
   "Databases" => [
     %{
       "Arn" => "[masked]",
       "CreationTime" => 1685091519.535,
       "DatabaseName" => "[masked]",
       "KmsKeyId" => "[masked]",
       "LastUpdatedTime" => 1716462052.933,
       "TableCount" => 12
     }
   ]
 },
 %{
   <omission>
 }}
  • 🎉1
  • ❤️1