rackerlabs/lambda-uploader

Requirements not being copied to zip

Closed this issue · 4 comments

When I specify pip libraries in the config or requirements.txt It looks like it installs it but does not copy to the zip file

Is there something I'm doing wrong? I have attached the zip, where there is no lib files

lambda_function.zip

λ Building Package
INFO:lambda_uploader.package:Building new virtualenv and installing requirements
DEBUG:lambda_uploader.package:Virtualenv stdout: Using real prefix '/usr'
New python executable in /home/testuser/py/ca-lambda/test/.lambda_uploader_temp/venv/bin/python2
Also creating executable in /home/testuser/py/ca-lambda/test/.lambda_uploader_temp/venv/bin/python
Installing setuptools, pip, wheel...done.
Running virtualenv with interpreter /home/testuser/py/env2/bin/python2

DEBUG:lambda_uploader.package:Virtualenv stderr:
DEBUG:lambda_uploader.package:Installing requirements found [u'boto3', u'requests', u'validators', u'Jinja2'] in config
DEBUG:lambda_uploader.package:Pip stdout: Requirement already satisfied: boto3 in /home/testuser/py/env2/lib/python2.7/dist-packages
Requirement already satisfied: requests in /home/testuser/py/env2/lib/python2.7/dist-packages
Requirement already satisfied: validators in /home/testuser/py/env2/lib/python2.7/dist-packages
Requirement already satisfied: Jinja2 in /home/testuser/py/env2/lib/python2.7/dist-packages
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /home/testuser/py/env2/lib/python2.7/dist-packages (from boto3)
Requirement already satisfied: botocore<1.5.0,>=1.4.1 in /home/testuser/py/env2/lib/python2.7/dist-packages (from boto3)
Requirement already satisfied: s3transfer<0.2.0,>=0.1.10 in /home/testuser/py/env2/lib/python2.7/dist-packages (from boto3)
Requirement already satisfied: decorator>=3.4.0 in /home/testuser/py/env2/lib/python2.7/dist-packages (from validators)
Requirement already satisfied: six>=1.4.0 in /home/testuser/py/env2/lib/python2.7/dist-packages (from validators)
Requirement already satisfied: MarkupSafe>=0.23 in /home/testuser/py/env2/lib/python2.7/dist-packages (from Jinja2)
Requirement already satisfied: docutils>=0.10 in /home/testuser/py/env2/lib/python2.7/dist-packages (from botocore<1.5.0,>=1.4.1->boto3)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /home/testuser/py/env2/lib/python2.7/dist-packages (from botocore<1.5.0,>=1.4.1->boto3)
Requirement already satisfied: futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" in /home/testuser/py/env2/lib/python2.7/dist-packages (from s3transfer<0.2.0,>=0.1.10->boto3)

DEBUG:lambda_uploader.package:Pip stderr:
INFO:lambda_uploader.package:Copying site packages
INFO:lambda_uploader.utils:Copying source files
INFO:lambda_uploader.package:Copying lib64 site packages
INFO:lambda_uploader.utils:Copying source files
INFO:lambda_uploader.utils:Copying source files
DEBUG:lambda_uploader.utils:Copying /home/testuser/py/ca-lambda/test/lambda.json to /home/testuser/py/ca-lambda/test/.lambda_uploader_temp/lambda_package/
DEBUG:lambda_uploader.utils:Copying /home/testuser/py/ca-lambda/test/hello.py to /home/testuser/py/ca-lambda/test/.lambda_uploader_temp/lambda_package/
DEBUG:lambda_uploader.utils:Copying /home/testuser/py/ca-lambda/test/requirements.txt to /home/testuser/py/ca-lambda/test/.lambda_uploader_temp/lambda_package/
INFO:lambda_uploader.package:Creating zipfile
DEBUG:lambda_uploader.package:Zipping /home/testuser/py/ca-lambda/test/.lambda_uploader_temp/lambda_package/lambda.json as lambda.json
DEBUG:lambda_uploader.package:Zipping /home/testuser/py/ca-lambda/test/.lambda_uploader_temp/lambda_package/hello.py as hello.py
DEBUG:lambda_uploader.package:Zipping /home/testuser/py/ca-lambda/test/.lambda_uploader_temp/lambda_package/requirements.txt as requirements.txt```




Hello! It seems like you have a lot of packages that pip is refusing to install because they are installed globally in your Python (see the part about Requirement already satisfied). You probably want to ship your entire set of installed Python libraries as a Lambda function, so I'd recommend uninstalling them globally so pip can install them locally for the project and virtualenv can capture them.

Hi martin,

Thanks for responding. env2 is a virtualenv not global

Hi @wobeng -- /home/testuser/py/ca-lambda/test/.lambda_uploader_temp/venv is a brand new virtualenv, so it shouldn't be able to see env2 unless you're actively loading it in all new shells/somehow making it globally activated. Is it possible you've got it loading automatically?

Lambda-uploader doesn't do anything beyond running virtualenv to create a separate environment, so for some reason, additional virtual environments aren't letting you install your dependencies. You could troubleshoot this yourself by attempting the same...

I had to rebuilt my linux environment, worked after that