Silent failure with dockerizePip on CI
timnoinc opened this issue · 2 comments
Are you certain it's a bug?
- Yes, it looks like a bug
Are you using the latest plugin release?
- Yes, I'm using the latest plugin release
Is there an existing issue for this?
- I have searched existing issues, it hasn't been reported yet
Issue description
potentially similar to #785
When I do serverless package --verbose -c serverless.yml -p dist --stage qa
, it works locally, but in Github Actions and in Jenkins the docker command errors out but doesn't seem to include any error message.
It's clear that something has gone wrong, but I haven't been able to find any log files anywhere to find out what. I've removed a good chunk of the functions and configuration in my serverless.yml, but it still happens consistently with this serverless.yml.
I need to package individually for size reasons. I need pip to run in docker because it's compiling science packages.
Service configuration (serverless.yml) content
service: projectName
frameworkVersion: '3'
provider:
name: aws
runtime: python3.10
environment:
S3_BUCKET: projectBucket
DEBUG: true
plugins:
- serverless-python-requirements
# - serverless-step-functions
custom:
pythonRequirements:
dockerizePip: true
functions:
Postback:
module: postback
handler: postback.lambda_handler
package:
individually: true
exclude:
- .git/**
- node_modules/**
Command name and used flags
package --verbose -c serverless.yml -p dist --stage qa
Command output
Run serverless/github-action@v3.2
/usr/bin/docker run --name e519d1f229f1572e3f48b3a1f7801d30a01c33_ab7ec9 --label e519d1 --workdir /github/workspace --rm -e "SLACK_WEBHOOK_URL" -e "DEPLOY_ENVIRONMENT" -e "DEPLOY_DESCRIPTION" -e "DEPLOYMENT_ID" -e "AWS_DEFAULT_REGION" -e "AWS_REGION" -e "AWS_ACCESS_KEY_ID" -e "AWS_SECRET_ACCESS_KEY" -e "AWS_SESSION_TOKEN" -e "NPM_CONFIG_USERCONFIG" -e "NODE_AUTH_TOKEN" -e "pythonLocation" -e "PKG_CONFIG_PATH" -e "Python_ROOT_DIR" -e "Python2_ROOT_DIR" -e "Python3_ROOT_DIR" -e "LD_LIBRARY_PATH" -e "INPUT_ARGS" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_ID_TOKEN_REQUEST_URL" -e "ACTIONS_ID_TOKEN_REQUEST_TOKEN" -e "ACTIONS_RESULTS_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/projectName/projectName":"/github/workspace" e519d1:f229f1572e3f48b3a1f7801d30a01c33 package --verbose -c serverless.yml -p dist --stage qa --param="deployDescription=develop-latest"
Running "serverless" from node_modules
Packaging projectName for stage qa (us-east-1)
Generated requirements from /github/workspace/postback/requirements.txt in /github/workspace/.serverless/postback/requirements.txt
Installing requirements from "/github/home/.cache/serverless-python-requirements/e29a8fbd0127f5ea361fb998fc6a150bec13d9d834f2dad4fb4e6a5d8497bb71_x86_64_slspyc/requirements.txt"
Docker Image: public.ecr.aws/sam/build-python3.10:latest-x86_64
Using download cache directory /github/home/.cache/serverless-python-requirements/downloadCacheslspyc
Running docker run --rm -v /github/home/.cache/serverless-python-requirements/e29a8fbd0127f5ea361fb998fc6a150bec13d9d834f2dad4fb4e6a5d8497bb71_x86_64_slspyc\:/var/task\:z -v /github/home/.cache/serverless-python-requirements/downloadCacheslspyc\:/var/useDownloadCache\:z public.ecr.aws/sam/build-python3.10\:latest-x86_64 /bin/sh -c 'chown -R 0\\:0 /var/useDownloadCache && python3.10 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache && chown -R 0\\:0 /var/task && chown -R 0\\:0 /var/useDownloadCache'...
Environment: linux, node 18.17.1, framework 3.36.0 (local) 3.36.0v (global), plugin 7.1.0, SDK 4.4.0
Error:
Docs: docs.serverless.com
Support: forum.serverless.com
Bugs: github.com/serverless/serverless/issues
1 deprecation found: run 'serverless doctor' for more details
Running "docker run --rm -v /github/home/.cache/serverless-python-requirements/e29a8fbd0127f5ea361fb998fc6a150bec13d9d834f2dad4fb4e6a5d8497bb71_x86_64_slspyc:/var/task:z -v /github/home/.cache/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z public.ecr.aws/sam/build-python3.10:latest-x86_64 /bin/sh -c chown -R 0\:0 /var/useDownloadCache && python3.10 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache && chown -R 0\:0 /var/task && chown -R 0\:0 /var/useDownloadCache" failed with: ""
Environment information
Running "serverless" from node_modules
Framework Core: 3.36.0 (local) 3.36.0 (global)
Plugin: 7.1.0
SDK: 4.4.0
`"serverless-python-requirements": "^6.0.1",` in my package.json
@timnoinc @kvnn Are you using the serverless github action by any chance?
I found that kept hitting silent errors when usin dockerizePip but also was using the above action. That action was erroring with Node 18 which was mismatched from the version I'd installed within setup-node. On closer inspection that actions Dockerfile is using Node 18 and Python 3.10.
Replacing it with an action that installs serverless seems to have resolved my issues:
- run: npm install -g serverless
- name: serverless deploy
run: "serverless plugin install -n serverless-python-requirements && serverless deploy --stage=prod --verbose"