Bug: Broken build when using sam/build-python3.10:latest-x86_64
dvernon83 opened this issue · 1 comments
Are you certain it's a bug?
- Yes, it looks like a bug
Are you using the latest plugin release?
- Yes, I'm using the latest plugin release
Is there an existing issue for this?
- I have searched existing issues, it hasn't been reported yet
Issue description
I made an issue for this with: aws/aws-sam-cli#7942
I am using Github Actions to automate my build. I deploy via serverless to AWS. My new build for yesterday broke pointed to public.ecr.aws/sam/build-python3.10:latest-x86_64. So I went to check on this and seen an update to the image. I then tried pinning to an older version and that didn't work. So I re-ran an older build that was successfully built and deployed on March 10th 3:09 pm CDT and that is now failing to build. Please advise. I have tried to build a docker image with the packages I need and it still fails. This is related to xmlsec1 library. I believe this is related to the image updates https://github.com/aws/aws-sam-cli did but I figured I would reach out here. Is there another image that would work with this plugin?
My python packages:
simplejson==3.16.0 mysql-connector-python==8.0.16 requests>=2.18.4 urllib3<2 Jinja2==3.1.4 python3-saml==1.16.0
Error Output:
Installing requirements from "/home/runner/.cache/serverless-python-requirements/f0f7c4fc76a31f7b165589bbaccd586c5f7dd8d9c33566b6b938bc221aa5ee19_x86_64_slspyc/requirements.txt"
Installing requirements
Building custom docker image from Dockerfile
Docker Image: sls-py-reqs-custom
Using download cache directory /home/runner/.cache/serverless-python-requirements/downloadCacheslspyc
Running docker run --rm -v /home/runner/.cache/serverless-python-requirements/f0f7c4fc76a31f7b165589bbaccd586c5f7dd8d9c33566b6b938bc221aa5ee19_x86_64_slspyc\:/var/task\:z -v /home/runner/.cache/serverless-python-requirements/downloadCacheslspyc\:/var/useDownloadCache\:z sls-py-reqs-custom /bin/sh -c 'chown -R 0\\:0 /var/useDownloadCache && python3.10 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache && chown -R 1001\\:118 /var/task && chown -R 1001\\:118 /var/useDownloadCache && find /var/task -name \\*.so -exec strip \\{\\} \\;'...
Finalizing
✖ ServerlessError2: Running "docker run --rm -v /home/runner/.cache/serverless-python-requirements/f0f7c4fc76a31f7b165589bbaccd586c5f7dd8d9c33566b6b938bc221aa5ee19_x86_64_slspyc:/var/task:z -v /home/runner/.cache/serverless-python-requirements/downloadCacheslspyc:/var/useDownloadCache:z sls-py-reqs-custom /bin/sh -c chown -R 0\:0 /var/useDownloadCache && python3.10 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache && chown -R 1001\:118 /var/task && chown -R 1001\:118 /var/useDownloadCache && find /var/task -name \*.so -exec strip \{\} \;" failed with: "error: subprocess-exited-with-error
× Building wheel for xmlsec (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [29 lines of output]
WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
Traceback (most recent call last):
File "/tmp/pip-build-env-y2724bgg/overlay/lib/python3.10/site-packages/setuptools_scm/_integration/pyproject_reading.py", line 36, in read_pyproject
section = defn.get("tool", {})[tool_name]
KeyError: 'setuptools_scm'
/tmp/pip-build-env-y2724bgg/overlay/lib/python3.10/site-packages/setuptools/dist.py:7[51](https://github.com/Market-Advisory-Group/market-nexus/actions/runs/13958840613/job/39076132070#step:12:52): SetuptoolsDeprecationWarning: License classifiers are deprecated.
********************************************************************************
Please consider removing the following classifiers in favor of a SPDX license expression:
License :: OSI Approved :: MIT License
See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
********************************************************************************
self._finalize_license_expression()
running bdist_wheel
running build
running build_py
creating build/lib.linux-x86_64-cpython-310/xmlsec
copying src/xmlsec/tree.pyi -> build/lib.linux-x86_64-cpython-310/xmlsec
copying src/xmlsec/template.pyi -> build/lib.linux-x86_64-cpython-310/xmlsec
copying src/xmlsec/constants.pyi -> build/lib.linux-x86_64-cpython-310/xmlsec
copying src/xmlsec/__init__.pyi -> build/lib.linux-x86_64-cpython-310/xmlsec
copying src/xmlsec/py.typed -> build/lib.linux-x86_64-cpython-310/xmlsec
running build_ext
error: xmlsec1 is not installed or not in path.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for xmlsec
ERROR: Could not build wheels for xmlsec, which is required to install pyproject.toml-based projects
Notice: A new release of pip is available: 23.0.1 -> 25.0.1
Notice: To update, run: pip install --upgrade pip"
at installRequirements (/home/runner/work/market-***/market-***/node_modules/serverless-python-requirements/lib/pip.js:427:17)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async installRequirementsIfNeeded (/home/runner/work/market-***/market-***/node_modules/serverless-python-requirements/lib/pip.js:704:3)
at async ServerlessPythonRequirements.installAllRequirements (/home/runner/work/market-***/market-***/node_modules/serverless-python-requirements/lib/pip.js:782:29)
Service configuration (serverless.yml) content
org: redacted
app: redacted
service: redacted
plugins:
- serverless-python-requirements
- serverless-plugin-scripts
- serverless-dotenv-plugin
- serverless-offline # offline and other offline plugins must be last in this list
useDotenv: true
frameworkVersion: '4'
provider:
name: aws
deploymentMethod: direct
runtime: python3.10
stage: ${opt:stage, 'dev'}
region: ${self:custom.region.${self:provider.stage}}
profile: ${env:AWS_PROFILE, ''}
environment:
RDS_DB: ${self:custom.rds.db.${self:provider.stage}}
RDS_USER: ${self:custom.rds.user.${self:provider.stage}}
RDS_HOST: ${self:custom.rds.host.${self:provider.stage}.write}
RDS_SSM: ${self:custom.rds.ssm.${self:provider.stage}}
ENV: ${self:provider.stage}
LAYER_PATH: ${self:custom.layerParams.layerPath.${self:provider.stage}}
BASE_PATH: ${self:custom.basePath.${self:provider.stage}}
iam:
role:
statements:
- Effect: Allow
Action:
- "execute-api:ManageConnections"
- "execute-api:Invoke"
Resource:
- "*"
- Effect: Allow
Action:
- ssm:GetParameter
- ssm:GetParameters
Resource:
- '*'
- Effect: Allow
Action:
- 's3:PutObject'
- 's3:DeleteObject'
- 's3:PutObjectAcl'
- 's3:GetObject'
- 's3:ListBucket'
Resource:
- arn:aws:s3:::*/*
- Effect: 'Allow'
Action:
- 'ec2:DescribeNetworkInterfaces'
- 'ec2:CreateNetworkInterface'
- 'ec2:DeleteNetworkInterface'
- 'ec2:DescribeInstances'
- 'ec2:AttachNetworkInterface'
Resource: '*'
- Effect: Allow
Action:
- 'lambda:InvokeFunction'
- 'iam:PassRole'
- 'secretsmanager:GetSecretValue'
Resource:
- '*'
custom:
scripts:
versionHookCommand: > # the order of the params here matters. DO NOT change unless you know what you're doing.
python3 scripts/version_hook.py
${self:custom.rds.db.${self:provider.stage}} ${self:custom.rds.user.${self:provider.stage}} ${self:custom.rds.host.${self:provider.stage}.write} ${self:custom.rds.ssm.${self:provider.stage}}
${self:custom.rds.db.${self:custom.envHierarchy.${self:provider.stage}}} ${self:custom.rds.user.${self:custom.envHierarchy.${self:provider.stage}}}
${self:custom.rds.host.${self:custom.envHierarchy.${self:provider.stage}}.write} ${self:custom.rds.ssm.${self:custom.envHierarchy.${self:provider.stage}}} ${self:provider.stage}
${param:semVer, 'patch'} ${param:versionOverride, 'false'} ${param:mergeDeploy, 'false'}
hooks: # add other hooks here if you have more hook types that need to run the versionHookCommand
before:deploy:deploy: ${self:custom.scripts.versionHookCommand}
before:offline:start: ${self:custom.scripts.versionHookCommand}
# use command 'serverless requirements cleanCache' if you change requirements.txt
# and the versions are not updating. Sometimes the cache needs cleared out
serverless-offline:
# Set the prefix to an empty string to avoid the stage being added
noPrependStageInUrl: true
httpPort: ${env:SERVERLESS_PORT}
pythonRequirements:
dockerizePip: true
# dockerImage: sls-py-reqs-custom
# dockerImage: public.ecr.aws/sam/build-python3.10:latest-arm64
slim: true
slimPatternsAppendDefaults: false
slimPatterns:
- '**/*.py[c|o]'
- '**/__pycache__*'
region:
local: us-east-1
dev: us-east-1
stage: us-east-1
prod: us-east-1
envHierarchy:
local: local
dev: dev
stage: dev
prod: stage
basePath:
local: ${env:BASE_PATH}
dev: "/var/task"
stage: "/var/task"
prod: "/var/task"
zrok:
local: true
dev: false
stage: false
prod: false
saml:
cert:
idp:
ssm:
local: ""
dev: ${env:SAML_SSM_IDP_CERT_PATH}
prod: ""
path:
local: ${env:SAML_IDP_CERT_PATH}
dev: ""
prod: ""
sp:
ssm:
local: ""
dev: ${env:SAML_SSM_SP_CERT_PATH}
prod: ""
path:
local: ${env:SAML_SP_CERT_PATH}
dev: ""
prod: ""
key:
sp:
ssm:
local: ""
dev: ${env:SAML_SSM_SP_KEY_PATH}
prod: ""
path:
local: ${env:SAML_SP_KEY_PATH}
dev: ""
prod: ""
config:
local: ${env:SAML_CONFIG_PATH}
dev: ${env:SAML_CONFIG_PATH}
prod: ""
entityId:
local: ${env:SAML_ENTITY_ID}
dev: ${env:SAML_ENTITY_ID}
prod: ""
host:
local: ${env:SAML_HOST_DN}
dev: ${env:SAML_HOST_DN}
prod: ""
reply:
login:
local: ${env:SAML_ASSERTION_URL}
dev: ${env:SAML_ASSERTION_URL}
prod: ""
logout:
local: ${env:SAML_SLO_URL}
dev: ${env:SAML_SLO_URL}
prod: ""
layerParams:
layerRef: # local does not need a value as we directly load the layer module via appending to sys.path
nexus:
local: {}
dev: { Ref: NexusLambdaLayer }
stage: { Ref: NexusLambdaLayer }
prod: { Ref: NexusLambdaLayer }
layerPath: # needed for local offline layers, for now, only local needs a value
local: "layers/python/lib/python3.10/site-packages"
dev: ""
stage: ""
prod: ""
rds:
ssm:
local: ${env:RDS_SSM}
dev: ${env:RDS_SSM}
stage: ${env:RDS_SSM}
prod: ${env:RDS_SSM}
host:
local:
read: ${env:RDS_HOST_READ}
write: ${env:RDS_HOST_WRITE}
dev:
read: ${env:RDS_HOST_READ}
write: ${env:RDS_HOST_WRITE}
stage:
read: ${env:RDS_HOST_READ}
write: ${env:RDS_HOST_WRITE}
prod:
read: ${env:RDS_HOST_READ}
write: ${env:RDS_HOST_WRITE}
user:
local: ${env:RDS_USER}
dev: ${env:RDS_USER}
stage: ${env:RDS_USER}
prod: ${env:RDS_USER}
db:
local: ${env:RDS_DB}
dev: ${env:RDS_DB}
stage: ${env:RDS_DB}
prod: ${env:RDS_DB}
templates:
path:
local: "templates"
dev: "templates"
stage: "templates"
prod: "templates"
static:
path:
local: ${env:STATIC_PATH}
dev: ${env:STATIC_PATH}
stage: ${env:STATIC_PATH}
prod: ${env:STATIC_PATH}
session:
duration:
default:
local: "43200"
dev: "43200"
stage: "43200"
prod: "43200"
azure:
app:
api:
token:
url:
local: "https://login.microsoftonline.com"
dev: "https://login.microsoftonline.com"
stage: "https://login.microsoftonline.com"
prod: "https://login.microsoftonline.com"
graph:
url:
local: "https://graph.microsoft.com"
dev: "https://graph.microsoft.com"
stage: "https://graph.microsoft.com"
prod: "https://graph.microsoft.com"
client_id:
local: ${env:AZURE_CLIENT_ID}
dev: ${env:AZURE_CLIENT_ID}
stage: ${env:AZURE_CLIENT_ID}
prod: ${env:AZURE_CLIENT_ID}
tenant_id:
local: ${env:AZURE_TENANT_ID}
dev: ${env:AZURE_TENANT_ID}
stage: ${env:AZURE_TENANT_ID}
prod: ${env:AZURE_TENANT_ID}
group_id:
local: ${env:AZURE_GROUP_ID}
dev: ${env:AZURE_GROUP_ID}
stage: ${env:AZURE_GROUP_ID}
prod: ${env:AZURE_GROUP_ID}
secret:
local: ${env:AZURE_CLIENT_SECRET}
dev: ${env:AZURE_CLIENT_SECRET}
stage: ${env:AZURE_CLIENT_SECRET}
prod: ${env:AZURE_CLIENT_SECRET}
layers:
nexus:
path: layers
compatibleRuntimes:
- python3.10
package:
# individually: true
exclude:
- "db/**"
- "node_modules/**"
- "static/**"
- "package-lock.json"
- "README.md"
functions:
# # # # # # # # # # # # # # # # # #
# #
# BACKEND LAMBDAS START HERE #
# #
# # # # # # # # # # # # # # # # # #
backend-auth:
handler: functions/api/auth/lib/handler.main
memorySize: 1024
timeout: 30
environment:
SAML_IDP_CERT_PATH: ${self:custom.saml.cert.idp.path.${self:provider.stage}}
SAML_SP_CERT_PATH: ${self:custom.saml.cert.sp.path.${self:provider.stage}}
SAML_SP_KEY_PATH: ${self:custom.saml.key.sp.path.${self:provider.stage}}
SAML_SSM_IDP_CERT_PATH: ${self:custom.saml.cert.idp.ssm.${self:provider.stage}}
SAML_SSM_SP_CERT_PATH: ${self:custom.saml.cert.sp.ssm.${self:provider.stage}}
SAML_SSM_SP_KEY_PATH: ${self:custom.saml.key.sp.ssm.${self:provider.stage}}
SAML_CONFIG_PATH: ${self:custom.saml.config.${self:provider.stage}}
SAML_ENTITY_ID: ${self:custom.saml.entityId.${self:provider.stage}}
SAML_HOST_DN: ${self:custom.saml.host.${self:provider.stage}}
SAML_ASSERTION_URL: ${self:custom.saml.reply.login.${self:provider.stage}}
SAML_SLO_URL: ${self:custom.saml.reply.logout.${self:provider.stage}}
SESSION_DURATION_DEFAULT: ${self:custom.session.duration.default.${self:provider.stage}}
AZURE_TOKEN_URL: ${self:custom.azure.app.api.token.url.${self:provider.stage}}
AZURE_GRAPH_URL: ${self:custom.azure.app.api.graph.url.${self:provider.stage}}
AZURE_CLIENT_ID: ${self:custom.azure.app.api.client_id.${self:provider.stage}}
AZURE_TENANT_ID: ${self:custom.azure.app.api.tenant_id.${self:provider.stage}}
AZURE_GROUP_ID: ${self:custom.azure.app.api.group_id.${self:provider.stage}}
AZURE_CLIENT_SECRET: ${self:custom.azure.app.api.secret.${self:provider.stage}}
ZROK_SKIP_HTTP: ${self:custom.zrok.${self:provider.stage}}
events:
- http:
path: login
method: post
cors: true
- http:
path: login/callback
method: post
cors: true
- http:
path: logout
method: post
cors: true
- http:
path: logout/callback
method: post
cors: true
- http:
path: slo/logout
method: post
cors: true
- http:
path: slo/logout/callback
method: get
cors: true
package:
exclude:
- ./**
include:
- 'functions/api/auth/lib/handler.py'
- 'functions/api/auth/venv/lib/python3.10/site-packages/**'
layers:
- ${self:custom.layerParams.layerRef.nexus.${self:provider.stage}}
backend-tabs:
handler: functions/api/tabs/lib/handler.main
memorySize: 1024
timeout: 30
environment:
STATIC_PATH: ${self:custom.templates.static.path.${self:provider.stage}}
SESSION_DURATION_DEFAULT: ${self:custom.session.duration.default.${self:provider.stage}}
ZROK_SKIP_HTTP: ${self:custom.zrok.${self:provider.stage}}
AZURE_TOKEN_URL: ${self:custom.azure.app.api.token.url.${self:provider.stage}}
AZURE_GRAPH_URL: ${self:custom.azure.app.api.graph.url.${self:provider.stage}}
AZURE_CLIENT_ID: ${self:custom.azure.app.api.client_id.${self:provider.stage}}
AZURE_TENANT_ID: ${self:custom.azure.app.api.tenant_id.${self:provider.stage}}
AZURE_GROUP_ID: ${self:custom.azure.app.api.group_id.${self:provider.stage}}
AZURE_CLIENT_SECRET: ${self:custom.azure.app.api.secret.${self:provider.stage}}
events:
- http:
path: account/settings
method: post
cors: true
- http:
path: admin/settings
method: post
cors: true
- http:
path: clients
method: post
cors: true
- http:
path: employee/nexus
method: post
cors: true
- http:
path: action/quick/look
method: post
cors: true
package:
exclude:
- ./**
include:
- 'functions/api/tabs/lib/handler.py'
- 'functions/api/tabs/includes/*'
- 'functions/api/tabs/venv/lib/python3.10/site-packages/**'
layers:
- ${self:custom.layerParams.layerRef.nexus.${self:provider.stage}}
backend-components:
handler: functions/api/components/lib/handler.main
memorySize: 1024
timeout: 30
environment:
STATIC_PATH: ${self:custom.templates.static.path.${self:provider.stage}}
SESSION_DURATION_DEFAULT: ${self:custom.session.duration.default.${self:provider.stage}}
ZROK_SKIP_HTTP: ${self:custom.zrok.${self:provider.stage}}
events:
- http:
path: component/modal/permission/edit/save
method: post
cors: true
- http:
path: component/modal/org/edit/save
method: post
cors: true
package:
exclude:
- ./**
include:
- 'functions/api/components/lib/handler.py'
- 'functions/api/components/includes/*'
- 'functions/api/components/venv/lib/python3.10/site-packages/**'
layers:
- ${self:custom.layerParams.layerRef.nexus.${self:provider.stage}}
# # # # # # # # # # # # # # # # # #
# #
# FRONTEND LAMBDAS START HERE #
# #
# # # # # # # # # # # # # # # # # #
frontend-auth:
handler: functions/frontend/auth/lib/handler.main
memorySize: 1024
timeout: 30
environment:
TEMPLATES_PATH: ${self:custom.templates.path.${self:provider.stage}}
STATIC_PATH: ${self:custom.templates.static.path.${self:provider.stage}}
SESSION_DURATION_DEFAULT: ${self:custom.session.duration.default.${self:provider.stage}}
ZROK_SKIP_HTTP: ${self:custom.zrok.${self:provider.stage}}
events:
- http:
path: login
method: get
cors: true
package:
exclude:
- ./**
include:
- 'functions/frontend/auth/lib/handler.py'
- 'functions/frontend/auth/venv/lib/python3.10/site-packages/**'
layers:
- ${self:custom.layerParams.layerRef.nexus.${self:provider.stage}}
frontend-dashboard:
handler: functions/frontend/dashboard/lib/handler.main
memorySize: 1024
timeout: 30
environment:
TEMPLATES_PATH: ${self:custom.templates.path.${self:provider.stage}}
STATIC_PATH: ${self:custom.templates.static.path.${self:provider.stage}}
SESSION_DURATION_DEFAULT: ${self:custom.session.duration.default.${self:provider.stage}}
ZROK_SKIP_HTTP: ${self:custom.zrok.${self:provider.stage}}
events:
- http:
path: dashboard
method: get
cors: true
package:
exclude:
- ./**
include:
- 'functions/frontend/dashboard/lib/handler.py'
- 'functions/frontend/dashboard/venv/lib/python3.10/site-packages/**'
layers:
- ${self:custom.layerParams.layerRef.nexus.${self:provider.stage}}
frontend-tabs:
handler: functions/frontend/tabs/lib/handler.main
memorySize: 1024
timeout: 30
environment:
TEMPLATES_PATH: ${self:custom.templates.path.${self:provider.stage}}
STATIC_PATH: ${self:custom.templates.static.path.${self:provider.stage}}
SESSION_DURATION_DEFAULT: ${self:custom.session.duration.default.${self:provider.stage}}
ZROK_SKIP_HTTP: ${self:custom.zrok.${self:provider.stage}}
AZURE_TOKEN_URL: ${self:custom.azure.app.api.token.url.${self:provider.stage}}
AZURE_GRAPH_URL: ${self:custom.azure.app.api.graph.url.${self:provider.stage}}
AZURE_CLIENT_ID: ${self:custom.azure.app.api.client_id.${self:provider.stage}}
AZURE_TENANT_ID: ${self:custom.azure.app.api.tenant_id.${self:provider.stage}}
AZURE_GROUP_ID: ${self:custom.azure.app.api.group_id.${self:provider.stage}}
AZURE_CLIENT_SECRET: ${self:custom.azure.app.api.secret.${self:provider.stage}}
events:
- http:
path: account/settings
method: get
cors: true
- http:
path: admin/settings
method: get
cors: true
- http:
path: clients
method: get
cors: true
- http:
path: employee/nexus
method: get
cors: true
- http:
path: action/quick/look
method: get
cors: true
package:
exclude:
- ./**
include:
- 'functions/frontend/tabs/lib/handler.py'
- 'functions/frontend/tabs/includes/*'
- 'functions/frontend/tabs/venv/lib/python3.10/site-packages/**'
layers:
- ${self:custom.layerParams.layerRef.nexus.${self:provider.stage}}
frontend-components:
handler: functions/frontend/components/lib/handler.main
memorySize: 1024
timeout: 30
environment:
TEMPLATES_PATH: ${self:custom.templates.path.${self:provider.stage}}
STATIC_PATH: ${self:custom.templates.static.path.${self:provider.stage}}
SESSION_DURATION_DEFAULT: ${self:custom.session.duration.default.${self:provider.stage}}
ZROK_SKIP_HTTP: ${self:custom.zrok.${self:provider.stage}}
events:
- http:
path: component/modal
method: get
cors: true
package:
exclude:
- ./**
include:
- 'functions/frontend/components/lib/handler.py'
- 'functions/frontend/components/includes/*'
- 'functions/frontend/components/venv/lib/python3.10/site-packages/**'
layers:
- ${self:custom.layerParams.layerRef.nexus.${self:provider.stage}}Command name and used flags
npx serverless deploy -s ${{ env.STAGE }} ${{ env.EXTRA_ARGS }} --debug
Command output
N/AEnvironment information
You're not getting my env file bruv. But you can see my github actions -> workflow.yaml
name: Deploy via Serverless Framework v4
on:
push:
branches:
- dev
- master
jobs:
determine-environment:
runs-on: ubuntu-latest
outputs:
env_name: ${{ steps.set-env.outputs.env_name }}
steps:
- name: Set environment name
id: set-env
run: |
if [ "${{ github.ref }}" == "refs/heads/dev" ]; then
echo "env_name=dev" >> $GITHUB_OUTPUT
elif [ "${{ github.ref }}" == "refs/heads/master" ]; then
echo "env_name=prod" >> $GITHUB_OUTPUT
fi
deploy:
runs-on: ubuntu-latest
needs: determine-environment
environment:
name: ${{ needs.determine-environment.outputs.env_name }}
steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v2
with:
node-version: '18'
- name: Install dependencies
run: npm install -f
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.10'
- name: Install Python dependencies
run: pip install mysql-connector-python==8.0.16 boto3
- name: Set environment variables
id: set-env
run: |
# we would only use this logic if the environments got weird or the extra args is different for a branch
# if [ "${{ needs.determine-environment.outputs.env_name }}" == "dev" ]; then
# export STAGE=dev
# export EXTRA_ARGS=--param="mergeDeploy=true"
# elif [ "${{ needs.determine-environment.outputs.env_name }}" == "prod" ]; then
# export STAGE=prod
# export EXTRA_ARGS=--param="mergeDeploy=true"
# fi
export STAGE=${{ needs.determine-environment.outputs.env_name }}
export EXTRA_ARGS=--param="mergeDeploy=true"
echo "STAGE=$STAGE" >> $GITHUB_ENV
echo "EXTRA_ARGS=$EXTRA_ARGS" >> $GITHUB_ENV
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: us-east-1
role-to-assume: ${{ secrets.AWS_ASSUME_ROLE_ARN }}
role-session-name: GithubActionsSession-${{ env.STAGE }}
role-duration-seconds: 3600
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- name: Generate CSS manifest.json
run: |
find static/css -type f -name "*.css" | jq -R . | jq -s '{"css_files": .}' > static/css/manifest.json
- name: Sync static files to S3
run: |
aws s3 sync ./static s3://redacted-${{ env.STAGE }}/static --acl bucket-owner-full-control --delete --exact-timestamps
- name: Invalidate CloudFront
run: |
aws cloudfront create-invalidation --distribution-id ${{ vars.CLOUDFRONT_STATIC_DISTRO_ID }} --paths "/*"
- name: Clean Serverless Python Requirements Cache
env:
SERVERLESS_ACCESS_KEY: ${{ secrets.SERVERLESS_ACCESS_KEY }}
SERVERLESS_PORT: ""
RDS_DB: ${{ vars.RDS_DB }}
RDS_HOST_READ: ${{ secrets.RDS_HOST_READ }}
RDS_HOST_WRITE: ${{ secrets.RDS_HOST_WRITE }}
RDS_SSM: ${{ secrets.RDS_SSM }}
RDS_USER: ${{ secrets.RDS_USER }}
SAML_IDP_CERT_PATH: ""
SAML_SP_CERT_PATH: ""
SAML_SP_KEY_PATH: ""
SAML_SSM_IDP_CERT_PATH: ${{ secrets.SAML_SSM_IDP_CERT_PATH }}
SAML_SSM_SP_CERT_PATH: ${{ secrets.SAML_SSM_SP_CERT_PATH }}
SAML_SSM_SP_KEY_PATH: ${{ secrets.SAML_SSM_SP_KEY_PATH }}
SAML_CONFIG_PATH: ${{ vars.SAML_CONFIG_PATH }}
SAML_ENTITY_ID: ${{ vars.SAML_ENTITY_ID }}
SAML_HOST_DN: ""
SAML_ASSERTION_URL: ${{ vars.SAML_ASSERTION_URL }}
SAML_SLO_URL: ${{ vars.SAML_SLO_URL }}
STATIC_PATH: ${{ vars.STATIC_PATH }}
BASE_PATH: ""
AZURE_CLIENT_ID: ${{ vars.AZURE_CLIENT_ID }}
AZURE_TENANT_ID: ${{ vars.AZURE_TENANT_ID }}
AZURE_GROUP_ID: ${{ vars.AZURE_GROUP_ID }}
AZURE_CLIENT_SECRET: ${{ secrets.AZURE_CLIENT_SECRET }}
ZROK_SKIP_HTTP: false
run: |
echo "Cleaning serverless-python-requirements cache..."
rm -rf ~/.cache/serverless-python-requirements
npx serverless requirements cleanCache
# - name: Check Docker installation
# run: |
# which docker
# docker --version
# - name: Build and Tag Custom SAM Build Image Inline
# run: |
# echo "Pulling base image..."
# docker pull public.ecr.aws/sam/build-python3.10:latest-x86_64
# echo "Running container to install xmlsec1..."
# CONTAINER_ID=$(docker run -d public.ecr.aws/sam/build-python3.10:latest-x86_64 sh -c "yum install -y libxml2-devel xmlsec1 xmlsec1-openssl && yum clean all && command -v xmlsec1 && xmlsec1 --version && sleep 30")
# docker wait $CONTAINER_ID
# echo "Committing container as 'sls-py-reqs-custom'..."
# docker commit $CONTAINER_ID sls-py-reqs-custom
# docker rm $CONTAINER_ID
# echo "Image 'sls-py-reqs-custom' built and tagged successfully."
# - name: Verify Custom Docker Image with Volume Mounts
# run: |
# echo "Creating dummy cache directories..."
# mkdir -p /home/runner/.cache/serverless-python-requirements/test_task
# mkdir -p /home/runner/.cache/serverless-python-requirements/test_cache
# echo "Running container from image 'sls-py-reqs-custom' with volume mounts..."
# docker run --rm \
# -v /home/runner/.cache/serverless-python-requirements/test_task:/var/task:z \
# -v /home/runner/.cache/serverless-python-requirements/test_cache:/var/useDownloadCache:z \
# sls-py-reqs-custom sh -c "echo 'Container PATH: ' \$PATH && command -v xmlsec1 && xmlsec1 --version"
# echo "Verification complete."
- name: Deploy with Serverless Framework v4
env:
SERVERLESS_ACCESS_KEY: ${{ secrets.SERVERLESS_ACCESS_KEY }}
SERVERLESS_PORT: ""
RDS_DB: ${{ vars.RDS_DB }}
RDS_HOST_READ: ${{ secrets.RDS_HOST_READ }}
RDS_HOST_WRITE: ${{ secrets.RDS_HOST_WRITE }}
RDS_SSM: ${{ secrets.RDS_SSM }}
RDS_USER: ${{ secrets.RDS_USER }}
SAML_IDP_CERT_PATH: ""
SAML_SP_CERT_PATH: ""
SAML_SP_KEY_PATH: ""
SAML_SSM_IDP_CERT_PATH: ${{ secrets.SAML_SSM_IDP_CERT_PATH }}
SAML_SSM_SP_CERT_PATH: ${{ secrets.SAML_SSM_SP_CERT_PATH }}
SAML_SSM_SP_KEY_PATH: ${{ secrets.SAML_SSM_SP_KEY_PATH }}
SAML_CONFIG_PATH: ${{ vars.SAML_CONFIG_PATH }}
SAML_ENTITY_ID: ${{ vars.SAML_ENTITY_ID }}
SAML_HOST_DN: ""
SAML_ASSERTION_URL: ${{ vars.SAML_ASSERTION_URL }}
SAML_SLO_URL: ${{ vars.SAML_SLO_URL }}
STATIC_PATH: ${{ vars.STATIC_PATH }}
BASE_PATH: ""
AZURE_CLIENT_ID: ${{ vars.AZURE_CLIENT_ID }}
AZURE_TENANT_ID: ${{ vars.AZURE_TENANT_ID }}
AZURE_GROUP_ID: ${{ vars.AZURE_GROUP_ID }}
AZURE_CLIENT_SECRET: ${{ secrets.AZURE_CLIENT_SECRET }}
ZROK_SKIP_HTTP: false
run: |
export PATH=/usr/bin:/usr/local/bin:$PATH
npx serverless deploy -s ${{ env.STAGE }} ${{ env.EXTRA_ARGS }} --debugDowngrading to pip package xmlsec==1.3.14 lets this build.