Issue with libpython2.7.so.1.0 when running on nodejs v10 runtime

Question

Issue with libpython2.7.so.1.0 when running on nodejs v10 runtime

renatoargh opened this issue 5 years ago · 30 comments

Hello,

I am getting an error with the following code:

const shell = require('child_process').execSync

module.exports.handler = () => {
    const version = shell('/opt/awscli/aws --version')
    console.log(version)
}

The error is as following: Command failed: /opt/awscli/aws --version\n/opt/awscli/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory\n

OBS: I got the layer by clicking this button:

What might have been wrong with my deployment?! Thank a lot for your work :)

Answer 1 · 2019-09-09T18:42:00.000Z

PS.: It works well with v8

Answer 2 · 2019-09-10T14:19:07.000Z

The aws-lambda-layer-awscli comes with /opt/awscli/bin/python which requires /lib64/libpython2.7.so.1.0, which unfortunately is not provided in nodejs10 lambda runtime.

bash-4.2# ldd /opt/awscli/bin/python
        linux-vdso.so.1 (0x00007ffd36dab000)
        libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007f1938d43000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1938b25000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f1938921000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007f193871e000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f19383d3000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f193801d000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1939120000)

However, /lib64/libpython2.7.so.1.0 is provided in the v8 runtime.

Answer 3 · 2019-09-10T15:34:13.000Z

Thanks @pahud
Do we plan to support it in the future?! Maybe by packaging some of these shared libraries?
My understanding is this regard is limited, otherwise, I could come up with a PR.

Answer 4 · 2019-09-11T00:04:50.000Z

@renatoargh Sure. Feel free to submit your PR.

Answer 5 · 2019-10-09T10:28:51.000Z

There is a PR somewhere ? Any idea to temporary solve this issue ?

Answer 6 · 2019-12-27T19:31:34.000Z

@pahud Has this issue been resolved? Is there a PR open for it?

Answer 7 · 2019-12-28T23:09:23.000Z

Hi,

I will leave this issue open and any PR would be appreciated.

Answer 8 · 2019-12-28T23:12:12.000Z

Just out of curiosity, as aws-lambda-layer-awscli is originally designed for lambda custom runtime with bash, is there any reason or scenario the nodejs runtime will use the aws CLI rather than the aws javascript sdk?

Answer 9 · 2019-12-30T12:29:19.000Z

We have one use case that leverages the aws s3 sync to copy the content of a directory to an s3 bucket. Apparently manually iterating over the directory contents and using the sdk to upload each file was too slow.

…

On Sat, Dec 28, 2019 at 18:12 Pahud Hsieh ***@***.***> wrote: Just out of curiosity, as aws-lambda-layer-awscli is originally designed for lambda custom runtime with bash, is there any reason or scenario the nodejs runtime will use the aws CLI rather than the aws javascript sdk? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5?email_source=notifications&email_token=ABDI6ZSXAN5EROTUZNCD2SLQ27MM3A5CNFSM4IU6T7Z2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHYT4RA#issuecomment-569458244>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDI6ZWTUL6Y34GOVCZUO7TQ27MM3ANCNFSM4IU6T7ZQ> .

Answer 10 · 2019-12-30T13:45:26.000Z

My use case here is exactly the same as @flesherj

Answer 11 · 2019-12-31T15:33:09.000Z

Hi,

This is expected behavior for the newer Lambda runtimes which use an Amazon Linux 2 base, which include only a very minimal set of shared libraries.

For the short term, you may want to try using one of the non-deprecated runtimes which use the older Amazon Linux 2018.03 base.

For example, with python3.6 here is a trivial example:

def handler(event, context):
  my_output = subprocess.check_output(["/opt/awscli/aws","--version"], stderr=subprocess.STDOUT, shell=False)
  print(my_output.decode('utf8'))

However, in the long term I also recommend using your own custom runtime.

Answer 12 · 2020-01-02T08:40:00.000Z

Hi, I have the same issue. I am using this Layer https://github.com/aws-samples/aws-lambda-layer-kubectl but both use AWSCLI. Perhaps the same fix can be applied to both.

I am using AWSCLI to call EKS update-config command as well as S3 sync commands. I am using Javascript for some logic.

Just an FYI: https://docs.aws.amazon.com/lambda/latest/dg/runtime-support-policy.html
Node 8 has reached EOL.

Answer 13 · 2020-01-02T10:52:05.000Z

Hi

Thanks for the feedback. I've successfully built an awscli layer that supports node12 lambda runtime

I will fork a new branch and share new assets in the next few hours.

Answer 14 · 2020-01-02T15:40:13.000Z

please check this new README section in the new node12-runtime-support branch

https://github.com/aws-samples/aws-lambda-layer-awscli/tree/node12-runtime-support#node10-and-node12-aws-lambda-runtime-support

Let me know whether it works before we merge it into master.

Thanks.

Answer 15 · 2020-04-20T14:05:14.000Z

I was working on AWS CLI v2 with python3 support for Lambda python3.8 runtime at #10 (comment)

And it seems to work with nodejs 12.x runtime as well

I am considering to start a new v2 branch to support AWS CLI v2 for all lambda runtimes.

Answer 16 · 2020-06-07T22:21:12.000Z

please check this new README section in the new node12-runtime-support branch
https://github.com/aws-samples/aws-lambda-layer-awscli/tree/node12-runtime-support#node10-and-node12-aws-lambda-runtime-support
Let me know whether it works ...

Thanks for your node12-runtime-support branch! I've followed the steps and attempted to use the layer for my Node 12 runtime. But found that the layer unzips to 98,588,610 bytes, ~~so unfortunately my use case happens to go just over the limit~~ Edit: managed to reduce my package size. Does the size sound right to you? (Thought to double-check in case this is unexpected.)

Edit: Ran the two make commands and added the env vars — works great!

Answer 17 · 2020-06-08T04:39:45.000Z

@logicalicy

Sounds like it works in your environemt. :-)

I just ran the make layer-build to generate the layer.zip. The zip file is about 12MB while the unzipped awscli dir is about 64MB.

You can customize your Dockerfile to generate the final layer.zip file.

Answer 18 · 2020-09-14T12:51:07.000Z

@pahud thanks for this, layer-build-python27 works well in Node 12 runtime. Will be great if we can have this in master.

Answer 19 · 2020-09-19T09:57:05.000Z

@vhadianto merged :-)

Answer 20 · 2020-09-21T06:06:45.000Z

@vhadianto merged :-)

Awesome, thank you very much :)

Answer 21 · 2020-11-13T11:46:49.000Z

Wonderful thanks! Works as expected.

Answer 22 · 2020-11-17T14:40:50.000Z

Hi @wlarch
Would you give me the example that works fine on your side?
I am trying to execute awscli within lambda, but still failed.
So I need your help.
Thanks

Answer 23 · 2020-11-17T14:51:31.000Z

@Venus713

Clone the repository (https://github.com/aws-samples/aws-lambda-layer-awscli/) and checkout the node12-runtime-support branch
Edit the Makefile to have the adequate configuration
Execute make build will build your layer in a Docker container and zip its content. Building the layer consists of following the steps of executing commands in the Dockerfile.python27 file.
Once the build is complete you can execute make layer-upload and then make layer-publish in order to upload the zipped layer to an S3 Bucket and then publish the layer in your AWS Lambda custom layers.
Configure the Lambda function to use your custom layer either by command line on directly in the AWS Lambda Console.

You can then execute /opt/awscli/aws --version in your nodejs12.x Lambda runtime using Nodejs child_process.

I will probably write a fully explained article shortly. I found that a lot of searching and experimentation is needed to understand this examples repository.

Answer 24 · 2020-11-17T15:19:43.000Z

@wlarch
Thanks, but I am still getting same issue.
If you are available, then would you guide me on my machine via TeamViewer?

Answer 25 · 2020-11-17T16:02:03.000Z

Hi @wlarch @pahud @vhadianto
Would you please review my issue?

I cloned the https://github.com/aws-samples/aws-lambda-layer-awscli/ and then execute make build command and S3BUCKET="awscli-layers-dev-serverlessdeploymentbucket-y5hlybvmyu8s" LAMBDA_REGION="us-east-1" make sam-layer-package sam-layer-deploy command. So I've got following result.

aws --region us-east-1 cloudformation describe-stacks --stack-name "awscli-layer-stack" --query 'Stacks[0].Outputs'
[
    {
        "OutputKey": "LayerVersionArn",
        "OutputValue": "arn:aws:lambda:us-east-1:412511290092:layer:awscli-layer-stack:3",
        "Description": "ARN for the published Layer version",
        "ExportName": "LayerVersionArn-awscli-layer-stack"
    }
]
[OK] Layer version deployed.

And then I did execute a lambda function in nodejs12.x version.

const shell = require('child_process').execSync

exports.handler = () => {
    const version = shell('/opt/awscli/aws --version')
    console.log(version)
}

In result, It gives me following error.

{
  "errorType": "Error",
  "errorMessage": "Command failed: /opt/awscli/aws --version\n/opt/awscli/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory\n",
  "trace": [
    "Error: Command failed: /opt/awscli/aws --version",
    "/opt/awscli/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory",
    "",
    "    at checkExecSyncError (child_process.js:630:11)",
    "    at execSync (child_process.js:666:15)",
    "    at Runtime.exports.handler (/var/task/index.js:4:21)",
    "    at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"
  ]
}

Request ID:
"bcadd745-01ba-43e9-b55d-2baac1da3cbd"
...

Do you have any suggestion for me?

Answer 26 · 2020-11-18T14:17:44.000Z

Hi @logicalicy
Did you read my comment?
Do you have any suggestion?

Answer 27 · 2020-11-18T14:41:02.000Z

@Venus713 I encountered the same problem before. Did you git checkout node12-runtime-support branch before building the layer and publishing it ?

Answer 28 · 2020-11-18T15:14:25.000Z

@wlarch
Yes, I did the git checkout node12-runtime-support as you said me.
Then I followed the ReadMe.

Answer 29 · 2021-01-11T02:19:13.000Z

Yes, I did the git checkout node12-runtime-support as you said me.
Then I followed the ReadMe.

@Venus713 have you ensured that your Lambda function uses the layer? Check in AWS Console.

Answer 30 · 2021-09-30T00:32:48.000Z

This also happens with dotnetcore3.1. any suggestions?