Error getting DynamoDb local latest tar.gz location undefined: 403

Question

Error getting DynamoDb local latest tar.gz location undefined: 403

maher-rddi opened this issue a year ago · 30 comments

Hi, I am getting Error: "Error getting DynamoDb local latest tar.gz location: 403" after doing installing serverless-dynamodb-local.

Here is the full description of the error message:

Started downloading dynamodb-local from http://s3-us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz into ***/.dynamodb. Process may take few minutes.

✖ Uncaught exception
Environment: darwin, node 16.17.0, framework 3.26.0 (local) 3.25.1v (global), plugin 6.2.2, SDK 4.3.2
Docs: docs.serverless.com
Support: forum.serverless.com
Bugs: github.com/serverless/serverless/issues

Error:
Error: Error getting DynamoDb local latest tar.gz location undefined: 403
at ClientRequest.
at Object.onceWrapper (node:events:628:26)
at ClientRequest.emit (node:events:513:28)
at ClientRequest.emit (node:domain:489:12)
at HTTPParser.parserOnIncomingClient [as onIncoming] (node:_http_client:674:27)
at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)
at Socket.socketOnData (node:_http_client:521:22)
at Socket.emit (node:events:513:28)
at Socket.emit (node:domain:489:12)
at addChunk (node:internal/streams/readable:315:12)
at readableAddChunk (node:internal/streams/readable:289:9)
at Socket.Readable.push (node:internal/streams/readable:228:10)
at TCP.onStreamRead (node:internal/stream_base_commons:190:23)

It seems that the bucket doesn't authorize my request!

Answer 1 · 2023-03-08T13:29:10.000Z

I'm getting the same error, apparently AWS changed the domain to https://s3.us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz

Edit:
The fix was done in dynamodb-localhost project but not published yet

Answer 2 · 2023-03-08T14:41:51.000Z

@matAlmeida Thanks, we will wait for the new release then.

Answer 3 · 2023-03-09T17:50:07.000Z

In the meantime, manually editing the download_url in node_modules/dynamodb-localhost/dynamodb/config.json to https://s3.us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz and the import of http to https in node_modules/dynamodb-localhost/dynamodb/installer.js should do the trick!

Answer 4 · 2023-03-09T18:01:43.000Z

@benjiJanssens it worked !! thanks a lot !

Answer 5 · 2023-03-13T15:28:18.000Z

@AshanFernando hey there! now that 99x/dynamodb-localhost#78 is merged, could we get a release to NPM which includes the fix?

Answer 6 · 2023-03-13T16:32:11.000Z

@cnuss you're in the wrong repo 😄 as soon as dynamodb-localhost gets a new release on npm, we can integrate the changes into serverless-dynamodb-local

Answer 7 · 2023-03-13T16:41:15.000Z

you're in the wrong repo

it appears 99x/serverless-dynamodb-local and 99x/dynamodb-localhost are maintained and released by the same individual(s) so I pinged here since this is the final(ish) destination 😄

Answer 8 · 2023-03-14T13:05:30.000Z

In the meantime, manually editing the download_url in node_modules/dynamodb-localhost/dynamodb/config.json to https://s3.us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz and the import of http to https in node_modules/dynamodb-localhost/dynamodb/installer.js should do the trick!

They should really make this configurable.

Answer 9 · 2023-03-22T22:22:25.000Z

any update on this issue? it's blocking our CI.

Answer 10 · 2023-03-24T16:27:29.000Z

any update on this issue? it's blocking our CI.

youre not alone :')

Answer 11 · 2023-03-27T09:48:38.000Z

Sad nothing happens :(

Answer 12 · 2023-03-28T01:10:28.000Z

In the meantime, manually editing the download_url in node_modules/dynamodb-localhost/dynamodb/config.json to https://s3.us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz and the import of http to https in node_modules/dynamodb-localhost/dynamodb/installer.js should do the trick!

It worked for me! Thanks!

Answer 13 · 2023-03-28T13:50:20.000Z

I've found a less-hacky way of getting the URL to update, and considering you're using NPM 8.3.0+ (also worked with yarn 1.22)

NOTE: I needed to clear my node_modules / build cache for this to work:

package.json:

  "devDependencies": {
      ...
      "serverless-dynamodb-local": "...",
      ...
  },
  "overrides": {
    "serverless-dynamodb-local": {
      "dynamodb-localhost": "https://github.com/99x/dynamodb-localhost#db30898f8c40932c7177be7b2f1a81360d12876d"
    }
  }

Ref: 99x/dynamodb-localhost#78
Ref: https://stackoverflow.com/questions/15806152/how-do-i-override-nested-npm-dependency-versions

Answer 14 · 2023-03-30T20:49:30.000Z

I had the same issue when trying to run a project. What helped me was changing two files from the node_modules/dynamodb-localhost folder.

In node_modules\dynamodb-localhost\dynamodb\installer.js change http to https:

node_modules\dynamodb-localhost\dynamodb\config.json change http to https

Maybe this will solve your problem

Answer 15 · 2023-03-31T09:07:00.000Z

Any ETA on when this is going to get published?

Answer 16 · 2023-03-31T14:55:31.000Z

To whom it may concern:

I faced this issue last week and CircleCI was really not liking it. I ended up adding the following in the scripts section of the packages.json file of the repo I was working with:

  "scripts": {
    "postinstall": "sed -i 's|http://s3-us-west-2|https://s3-us-west-2|g' ./node_modules/dynamodb-localhost/dynamodb/config.json && sed -i 's|require(\"http\")|require(\"https\")|g' ./node_modules/dynamodb-localhost/dynamodb/installer.js",
...
  },

Is it a bit hacky? yes, but I can get over it for now. I'm using sed to modify 2 files.

Answer 17 · 2023-03-31T16:45:01.000Z

@jespinal you could use patch-package for that

Edit: adding the link

Answer 18 · 2023-04-02T16:51:40.000Z

I've found a less-hacky way of getting the URL to update, and considering you're using NPM 8.3.0+ (also worked with yarn 1.22)

NOTE: I needed to clear my node_modules / build cache for this to work:

package.json:
  "devDependencies": {
      ...
      "serverless-dynamodb-local": "...",
      ...
  },
  "overrides": {
    "serverless-dynamodb-local": {
      "dynamodb-localhost": "https://github.com/99x/dynamodb-localhost#db30898f8c40932c7177be7b2f1a81360d12876d"
    }
  }
Ref: 99x/dynamodb-localhost#78 Ref: https://stackoverflow.com/questions/15806152/how-do-i-override-nested-npm-dependency-versions

I missed a line in my original comment, it also needs to be set in devDependencies for full integration

  "devDependencies": {
      ...
      "dynamodb-localhost": "https://github.com/99x/dynamodb-localhost#db30898f8c40932c7177be7b2f1a81360d12876d"
      ...
      "serverless-dynamodb-local": "...",
      ...
  },
  "overrides": {
    "serverless-dynamodb-local": {
      "dynamodb-localhost": "https://github.com/99x/dynamodb-localhost#db30898f8c40932c7177be7b2f1a81360d12876d"
    }
  }

Also found some issues with ourdering so make sure dynamodb-localhost is before serverless-dynamodb-local in devDependencies.

Answer 19 · 2023-04-04T04:38:37.000Z

I have a doubt, why using pnpm the folder dynamodb-localhost is not created? I can't do the trick to change download_url with pnpm.

Answer 20 · 2023-04-10T05:40:34.000Z

I fixed it by simply copying over the .dynamodb folder into the root from another project where I had already run sls dynamodb install before

Answer 21 · 2023-04-24T16:14:12.000Z

I have a doubt, why using pnpm the folder dynamodb-localhost is not created? I can't do the trick to change download_url with pnpm.

Doesn't work for me either

Answer 22 · 2023-05-08T18:19:48.000Z

Do we have fix this as of now ?

Answer 23 · 2023-05-08T20:43:39.000Z

man, this long and no fix? this is annoying

Answer 24 · 2023-05-09T12:23:03.000Z

man, this long and no fix? this is annoying

Last publish of the library was 2 years. Should we consider it as dead ?

Answer 25 · 2023-05-09T13:40:41.000Z

@ehtesham1996 Seems like it. I guess a fork would be a viable option

Answer 26 · 2023-05-31T06:20:14.000Z

In the meantime, manually editing the download_url in node_modules/dynamodb-localhost/dynamodb/config.json to https://s3.us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz and the import of http to https in node_modules/dynamodb-localhost/dynamodb/installer.js should do the trick!

thank you so much:
I fix it

Answer 27 · 2023-06-15T10:23:14.000Z

This issue appears to have been fixed in the latest release without need for any manual editing or switching the config json values

Answer 28 · 2023-06-17T13:41:51.000Z

This issue appears to have been fixed in the latest release without need for any manual editing or switching the config json values
On wich dependency is this fixe released?

The newest version via npm is serverless-dynamodb-local 0.2.40 and uses dynamodb-localhost 0.0.9 still with the old url.

As matAlmeida commented on Mar 8: PR 99x/dynamodb-localhost#78 merged but is not released yet.

In the comments are some suggestions provided on how to fix the issue:

benjidotsh commented on Mar 9 --> manually modify two files in node_modules/dynamodb-localhost and replace http to https
cnuss commented on Apr 2 --> add dynamodb-localhost from github containing the fix to dev dependencies
jespinal commented on Mar 31 --> add a postinstall script to package.json with sed commands to modify two files in node_modules/dynamodb-localhost and replace http to https

I used the command from jespinal:

sed -i 's|http://s3-us-west-2|https://s3-us-west-2|g' ./node_modules/dynamodb-localhost/dynamodb/config.json && sed -i 's|require(\"http\")|require(\"https\")|g' ./node_modules/dynamodb-localhost/dynamodb/installer.js

Answer 29 · 2023-07-04T00:36:43.000Z

FYI: This package (serverless-dynamodb-local) doesn't appear to be maintained, with no new releases to NPM in over 2 years. Despite our pleas, I don't think this is getting deployed anytime soon. Over time, I think solutions that require creative uses of patch-package or sed will become fragile, and are a pain to maintain on each repo.

Solution: You can use serverless-dynamodb, a maintained fork, instead. (Disclaimer: I am a contributor to this fork). It is a drop-in replacement for this package, and is updated to fix this bug. This is a drop-in replacement for serverless-dynamodb-local, so to upgrade simply:

Uninstall serverless-dynamodb-local, e.g. npm uninstall serverless-dynamodb-local
Install serverless-dynamodb, e.g. npm install serverless-dynamodb
Update references in your code, including your serverless config, from serverless-dynamodb-local to serverless-dynamodb (quite possible that you won't have any, as you just refer to it from your serverless config)
(optional) Update your serverless config custom dynamodb key to serverless-dynamodb

Of course, it's all still open-source and MIT licensed. Ownership of this new package sits with a registered charity, that is committed to maintaining the package into the future and is open to contributions from the community.

There's an explanation as to the intentions behind this fork, and how it compares to other forks in the README.

In any case would be open to feedback on the fork - can drop create an issue in that repository or email me (address on profile).

Answer 30 · 2023-11-23T00:08:50.000Z

Thank youu!