The request signature we calculated does not match the signature you provided

[villelahdenvuo]

I'm not sure what I'm doing wrong, but I'm getting this error.

Here's the code:

import { get } from 'lodash';
import { Client, ClientOptions } from '@elastic/elasticsearch';
import AWS from 'aws-sdk';

export function getCredentials() {
	return new AWS.CredentialProviderChain().resolvePromise();
}

export async function getElasticsearchClient() {
	const host = get(process.env, 'ELASTICSEARCH_HOST', '');
	const config: ClientOptions = { node };

	if (node.includes('es.amazonaws.com')) {
		config.Connection = require('aws-elasticsearch-connector');
		Object.defineProperty(config, 'awsConfig', {
			value: {
				credentials: await getCredentials()
			}
		});
	}

	return new Client(config);
};

// Usage:
const { body: mappings } = await es.indices.getMapping({
	index: 'my-index-*'
});

/*
{ ResponseError: Response Error
    at IncomingMessage.response.on (/node_modules/@elastic/elasticsearch/lib/Transport.js:287:25)
  name: 'ResponseError',
  meta:
   { body:
      { message:
         'The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.' },
     statusCode: 403,
     headers:
      { 'access-control-allow-origin': '*',
        'content-type': 'application/json',
        'x-amzn-requestid': 'bcdfb0d0-cd8d-11e9-9f14-9fd3554ba9dc',
        'content-length': '192',
        connection: 'keep-alive' },
     warnings: null,
     meta:
      { context: null,
        request: [Object],
        name: 'elasticsearch-js',
        connection: [Object],
        attempts: 0,
        aborted: false } } }
*/

Edit: I'm using "@elastic/elasticsearch": "^6.8.2" as the latest version AWS offers is 6.8, but I tried with 7.3.0 and I still get the same error. I made sure the AWS.config.credentials return correct values.

[compwright]

@villelahdenvuo thanks for the detailed info. What does await getCredentials() resolve to (redacted, of course)?

Could you try the integration test with your credentials for me and tell me if it works?

AWS_PROFILE=your-profile npm run test:integration -- --endpoint https://amazon-es-host.us-east-1.es.amazonaws.com

[villelahdenvuo]

I didn't have time to debug more so I reverted back to the old client. The credentials is an AWS.Config.Credentials object. I also tried passing the AWS.Config object directly with the same result. I logged it and it contains the correct credentials (the old client works the exact same way using the original package, so the credentials are okay).

I was logging the request made by the client and noticed that the Connection keep-alive header was not included in the signature. I tried to manually add it by changing the package code, but even with it added to the signature I got the same error so probably that's not the problem. But maybe something to consider.

If I find more time I will try to run the integrations tests.

[compwright]

@villelahdenvuo in light of your particular use case, I also looked into whether we could internally support resolving credentials asynchronously. But unfortunately the ES Connection class would have to support promise-based or callback-based request building, and it does not. One could in theory extend it, but you would probably also have to extend the ES Transport class. It would be messy and a maintenance nightmare to keep in sync with the upstream version.

So, this library won't be supporting asynchronous credentials directly anytime soon. The best approach is going to be the one you attempted, which is to resolve the credentials before initializing the ES client. If credentials need to be refreshed asynchronously, then wait for that to finish before making the next request and the client will read the new credentials from AWS.Config. I've added additional tests to verify that this does indeed work as intended.

[compwright]

Resolved by #2