aws/aws-sdk-js-v3

@aws-sdk/client-s3 makes node_modules too large

remagpie opened this issue · 6 comments

remagpie commented

Describe the bug

When @aws-sdk/client-s3@3.29.0 is installed, it downloads aws-crt as a dependency which is about 200MB in size. It occupies about 60MB even when we compress it.

My team updates the lambda function by directly zipping the directory and uploading it, but we can't do it anymore because we hit the size limit of the allowed archive size.

The problem is gone after we downgrade @aws-sdk/client-s3 to 3.28.0, but we'd like to use the most recent version if possible.

Your environment

SDK version number

@aws-sdk/client-s3@3.29.0

Is the issue in the browser/Node.js/ReactNative?

Node.js

Additional context

moltar commented

After bundling using esbuild and deploying to Lambda we are getting the following errors:

AWS CRT binary not present in any of the following locations:\n\t/bin/native/aws-crt-nodejs\n\t/bin/linux-x64/aws-crt-nodejs

    "stack": [
        "Error: AWS CRT binary not present in any of the following locations:",
        "\t/bin/native/aws-crt-nodejs",
        "\t/bin/linux-x64/aws-crt-nodejs",
        "    at null.node_modules/aws-crt/dist/native/binding.js (/node_modules/aws-crt/lib/native/binding.js:46:11)",
        "    at __require (/var/task/index.js:41:45)",
        "    at null.node_modules/aws-crt/dist/native/crt.js (/node_modules/aws-crt/lib/native/crt.ts:18:1)",
        "    at __require (/var/task/index.js:41:45)",
        "    at null.node_modules/aws-crt/dist/index.js (/node_modules/aws-crt/lib/index.ts:17:1)",
        "    at __require (/var/task/index.js:41:45)",
        "    at null.node_modules/@aws-sdk/signature-v4-crt/dist/cjs/CrtSignerV4.js (/node_modules/@aws-sdk/signature-v4-crt/src/CrtSignerV4.ts:22:1)",
        "    at __require (/var/task/index.js:41:45)",
        "    at null.node_modules/@aws-sdk/signature-v4-crt/dist/cjs/index.js (/node_modules/@aws-sdk/signature-v4-crt/src/index.ts:1:1)",
        "    at __require (/var/task/index.js:41:45)"
    ]
  • 👍4
ml1nk commented

In addition since aws-crt includes a native blob which does not support alpine, it fails for example on node:16.8.0-alpine.

ERROR: Error: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /opt/main/node_modules/aws-crt/dist/bin/linux-x64/aws-crt-nodejs.node)
  • 👍1
  • 👀4
webberwang commented

I notice Cannot find module '/node_modules/aws-crt/dist/bin/darwin-x64/aws-crt-nodejs' from '/node_modules/aws-crt/dist/native/binding.js' in 3.29.0 (which isn't fixed even if I install the missing package), had to downgrade to 3.28.0

matheuscanela commented

The s3 adds 209 packages as it gets removed after removing the client-s3
Cannot be used with lambda, as it consumes 200+ mb

  • 👀1
AllanZhengYP commented

Thank you for reporting this issue. We are investigating in the issue and will post update in #2750

github-actions commented

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.