Cryptodome with lambda AWS
Closed this issue · 26 comments
Hello,
Cryptodome works perfectly on my Windows 64 machine. I am trying to upload Cryptodome to a AWS lambda function but I am getting this error:
module initialization error: Cannot load native module 'Cryptodome.Cipher._raw_ecb'
What could be the issue?
Thank you!
It seems that lambda-packages: https://github.com/Miserlou/lambda-packages aims at making various packages lambda compatible.
Support is available for pycrypto, but not pycryptodome.
Checkout this issue: Miserlou/lambda-packages#48
I ran into the same problem while developing a lambda deployment package on Ubuntu machine, which came with Python 2.7.13 (python) and 3.5.3 (python3). I installed Python 3.6.1 (python3.6) and pip3.6 to reinstall pycryptodomex to my deployment package, and it's working. I can't seem to replicate my problem after having installed python3.6, so it may not be the right solution in some cases, but maybe it will work in your's, if you haven't already figured something else out.
Hi,
I would also like to see Cryptodone support for AWS Lambda, I am currently trying to use it in a deployment package. Has there been any movement on this?
Any news on this? I wanted to use it as a replacement for pyCrypto, but it breaks in Lambda.
Solved it by downgrading to Python 2.7, it would create problems with Python 3 on AWS Lambda.
Hi
I am using pyjks in my aws lambda function. pyjks internally dependent on pycryptodome
When i do "import jks" in my python script i am getting the below error. Please note that i have used virtualenv to package my lambda deployment to aws.
Please advise on this
_module initialization error: Cannot load native module 'Crypto.Hash._MD5': Trying '_MD5.cpython-36m-x86_64-linux-gnu.so': /var/task/Crypto/Util/../Hash/_MD5.cpython-36m-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory, Trying '_MD5.abi3.so': /var/task/Crypto/Util/../Hash/_MD5.abi3.so: cannot open shared object file: No such file or directory, Trying '_MD5.so': /var/task/Crypto/Util/../Hash/MD5.so: cannot open shared object file: No such file or directory
Mac and Linux VM builds dependencies differently. Try initializing and building the virtualenv on a linux vm.
I'm facing the same issue. I'm using venv and importing a module that relies on pycryptodome. Once I build the package and upload it to Lambda, it fails.
Going deeply on this, I started building the lambda package on Linux, I previously was doing it on MacOS. On Linux it went OK
Any news on this? The issue on my lambda is :
No such file or directory, Trying '_raw_ecb.so':
Struggling to find a way around this.
@rdlou Have you tried building on Linux? That's often the issue when .so files are missing.
I am on Linux... I think this is essentially my issue: https://stackoverflow.com/questions/54467095/module-initialization-error-cannot-load-native-module-crypto-cipher-raw-ecb
But it looks like the only person who got the answer to work ended up using 2.7, which I can't do. I'm looking at spinning it up on GCP what I'm making is just for a proof of concept. I still don't really understand why things work locally, but when the entire package is uploaded I have missing files.
Same problem here, and I would like to stay on 3.7 too. Did you manage to solve this ?
Nope, never solved it. Incredibly frustrating. My project has changed scope a little bit so it wasn't worth me pursuing.
Same here. The project has changed enough that now hashlib and passlib cover all my needs on the Python side of things.
Ok thanks for your feedback @rdlou @SpoonOfDoom. I just need a basic AES encryption, I have to find another way to do that.
Ok thanks for your feedback @rdlou @SpoonOfDoom. I just need a basic AES encryption, I have to find another way to do that.
If you find a good way to do that, I'd be interested to know, even if I have no immediate need for it.
To all people having problems:
Please confirm you are building the deployment package on Linux 64-bits (x86_64), directly or via Docker (e.g. image amazonlinux:latest) or VM.
Building it on any other OS (like OSX) will not work.
If you still have problems, please share exactly which Python version you use and how your build process roughly looks like.
Thanks @Legrandin, it's now working ! I was actually running it on Windows. I made a linux VM (Debian) and I installed it from the VM. Everything works perfectly in my Lambda now.
I hope it can help @SpoonOfDoom and @rdlou :)
In addition to @caillef note. Make sure you install PyCryptodome using the same Python version as you use in your Lambda function. I had python 3.7 locally while using 3.8 for Lambda. After I changed the Lambda runtime version to 3.7, I was able to make it work.
Ok thanks for your feedback @rdlou @SpoonOfDoom. I just need a basic AES encryption, I have to find another way to do that.
Were you able to find a way?
Also ran into this issue developing on a mac. Works when I build on a linux machine.
I spun up an t2.ec2-micro instance (Amazon Linux 2), and installed python3.8 from amazon-linux-extras.
Then I created a virtual environment, activated it, and pip installed pycryptodome.
I ziped the env/lib/python3.8/site-packages contents, and then scp'd this to my personal machine where I included it with my lambda deployment package.
This resolved the issue.
This should now be fixed as version 3.10.1 started using abi3 wheels.
Hey @Legrandin . The usage of abi3 in the latest version still doesn't solve the problem of using the package on AWS Lambda.
Instead I followed the approach of building the package on linux and then use it as a lambda layer and it works beautifully.
I installed it with amazonlinux:latest but it does not work.
Is this resolved?
Same issue here
"Cannot load native module 'Crypto.Hash._MD5': Not found '_MD5.cpython-39-aarch64-linux-gnu.so', Cannot load '_MD5.abi3.so': /opt/python/Crypto/Util/../Hash/_MD5.abi3.so: invalid ELF header, Not found '_MD5.so'"
I didn't try - but you may get some mileage using this technique
mkdir -p python/lib/python3.10/site-packages
echo "pycryptodome" > requirements.txt
sudo docker run -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.10:latest" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.10/site-packages/; exit"
zip -r lambda_function.zip .