Bug: SAM build ignores SkipBuild metadata flag when specifying the name of lambda
cj-mantas-gridinas opened this issue · 2 comments
cj-mantas-gridinas commented
Description:
Setting the skip build flag only affects lambda builds if you do not specify which lambda should be built using the build subcommand.
Steps to reproduce:
- Set
Resources.lambda.Metadata.SkipBuildto True - Run
sam build --template template.yml lambda
Example template.yml file
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: An AWS Serverless Application Model template describing your function.
Resources:
lambda:
Type: AWS::Serverless::Function
Metadata:
SkipBuild: True
Properties:
FunctionName: "foo"
CodeUri: .
Description: lambda with zip containing libraries as dedicated jars
MemorySize: 2048
Timeout: 600
Handler: foo.Main::handleRequest
Runtime: java21
Architectures:
- x86_64
EphemeralStorage:
Size: 1024
EventInvokeConfig:
MaximumEventAgeInSeconds: 21600
MaximumRetryAttempts: 2
DestinationConfig: {}
PackageType: Zip
SnapStart:
ApplyOn: None
RuntimeManagementConfig:
UpdateRuntimeOn: Auto
VpcConfig:
SecurityGroupIds:
- sg-1
SubnetIds:
- subnet-1
- subnet-2
Ipv6AllowedForDualStack: falseUsed the following build.gradle to show as proof that the build is still run.
plugins {
id 'java'
}
group = 'org.example'
version = '1.0-SNAPSHOT'
java {
sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21
}
repositories {
mavenCentral()
}
dependencies {
implementation "shit"
}
test {
useJUnitPlatform()
}
task buildZip(type: Zip) {
into('lib') {
from(jar)
from(configurations.runtimeClasspath)
}
}Observed result:
2024-09-24 13:33:33,417 | No config file found in this directory.
2024-09-24 13:33:33,419 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/samconfig.toml'
2024-09-24 13:33:33,420 | Config file location: /Users/mantas.gridinas/IdeaProjects/lambda-at-home/samconfig.toml
2024-09-24 13:33:33,420 | Config file '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/samconfig.toml' does not exist
2024-09-24 13:33:33,435 | OSError occurred while reading TOML file: [Errno 2] No such file or directory: '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/samconfig.toml'
2024-09-24 13:33:33,436 | Using config file: samconfig.toml, config environment: default
2024-09-24 13:33:33,436 | Expand command line arguments to:
2024-09-24 13:33:33,437 | --template_file=/Users/mantas.gridinas/IdeaProjects/lambda-at-home/template.yml --resource_logical_id=lambda --mount_with=READ --build_dir=.aws-sam/build
--cache_dir=.aws-sam/cache
2024-09-24 13:33:33,470 | 'build' command is called
2024-09-24 13:33:33,484 | No Parameters detected in the template
2024-09-24 13:33:33,500 | There is no customer defined id or cdk path defined for resource lambda, so we will use the resource logical id as the resource id
2024-09-24 13:33:33,501 | 0 stacks found in the template
2024-09-24 13:33:33,501 | No Parameters detected in the template
2024-09-24 13:33:33,513 | There is no customer defined id or cdk path defined for resource lambda, so we will use the resource logical id as the resource id
2024-09-24 13:33:33,514 | 1 resources found in the stack
2024-09-24 13:33:33,514 | Found Serverless function with name='lambda' and CodeUri='.'
2024-09-24 13:33:33,515 | --base-dir is not presented, adjusting uri . relative to /Users/mantas.gridinas/IdeaProjects/lambda-at-home/template.yml
2024-09-24 13:33:33,515 | Skip building pre-built function: lambda
2024-09-24 13:33:33,518 | 1 resources found in the stack
2024-09-24 13:33:33,519 | Found Serverless function with name='lambda' and CodeUri='.'
2024-09-24 13:33:33,519 | Skip building pre-built function: lambda
2024-09-24 13:33:33,520 | Instantiating build definitions
2024-09-24 13:33:33,520 | No previous build graph found, generating new one
2024-09-24 13:33:33,521 | Unique function build definition found, adding as new (Function Build Definition: BuildDefinition(java21, /Users/mantas.gridinas/IdeaProjects/lambda-at-home, Zip, ,
f30b6d8f-7c7b-4484-8908-bc74860671b0, {'SkipBuild': True}, {}, x86_64, []), Function: Function(function_id='lambda', name='lambda', functionname='foo', runtime='java21', memory=2048, timeout=600,
handler='foo.Main::handleRequest', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/Users/mantas.gridinas/IdeaProjects/lambda-at-home', environment=None, rolearn=None, layers=[], events=None,
metadata={'SkipBuild': True, 'SamResourceId': 'lambda'}, inlinecode=None, codesign_config_arn=None, architectures=['x86_64'], function_url_config=None, function_build_info=<FunctionBuildInfo.SkipBuild:
('SkipBuild', 'A Function which is denoted with SkipBuild in metadata, non buildable')>, stack_path='', runtime_management_config={'UpdateRuntimeOn': 'Auto'}, logging_config=None))
2024-09-24 13:33:33,523 | Building codeuri: /Users/mantas.gridinas/IdeaProjects/lambda-at-home runtime: java21 metadata: {'SkipBuild': True} architecture: x86_64 functions: lambda
2024-09-24 13:33:33,525 | Building to following folder /Users/mantas.gridinas/IdeaProjects/lambda-at-home/.aws-sam/build/lambda
2024-09-24 13:33:33,525 | Looking for a supported build workflow in following directories: ['/Users/mantas.gridinas/IdeaProjects/lambda-at-home', '/Users/mantas.gridinas/IdeaProjects/lambda-at-home']
2024-09-24 13:33:33,527 | Loading workflow module 'aws_lambda_builders.workflows'
2024-09-24 13:33:33,529 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
2024-09-24 13:33:33,530 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
2024-09-24 13:33:33,532 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
2024-09-24 13:33:33,534 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
2024-09-24 13:33:33,535 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
2024-09-24 13:33:33,537 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
2024-09-24 13:33:33,539 | Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)'
2024-09-24 13:33:33,541 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2024-09-24 13:33:33,543 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
2024-09-24 13:33:33,546 | Registering workflow 'RustCargoLambdaBuilder' with capability 'Capability(language='rust', dependency_manager='cargo', application_framework=None)'
2024-09-24 13:33:33,546 | Found workflow 'JavaGradleWorkflow' to support capabilities 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
2024-09-24 13:33:33,936 | Running workflow 'JavaGradleWorkflow'
2024-09-24 13:33:33,937 | Running JavaGradleWorkflow:GradleBuild
2024-09-24 13:33:34,937 | JavaGradleWorkflow:GradleBuild failed
Traceback (most recent call last):
File "aws_lambda_builders/workflows/java_gradle/actions.py", line 50, in _build_project
File "aws_lambda_builders/workflows/java_gradle/gradle.py", line 45, in build
aws_lambda_builders.workflows.java_gradle.gradle.GradleExecutionError: Gradle Failed: FAILURE: Build failed with an exception.
* Where:
Build file '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/build.gradle' line: 17
* What went wrong:
A problem occurred evaluating root project 'lambda-at-home'.
> Supplied String module notation 'shit' is invalid. Example notations: 'org.gradle:gradle-core:2.2', 'org.mockito:mockito-core:1.9.5:javadoc'.
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.
BUILD FAILED in 515ms
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "aws_lambda_builders/workflow.py", line 374, in run
File "aws_lambda_builders/workflows/java_gradle/actions.py", line 32, in execute
File "aws_lambda_builders/workflows/java_gradle/actions.py", line 58, in _build_project
aws_lambda_builders.actions.ActionFailedError: Gradle Failed: FAILURE: Build failed with an exception.
* Where:
Build file '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/build.gradle' line: 17
* What went wrong:
A problem occurred evaluating root project 'lambda-at-home'.
> Supplied String module notation 'shit' is invalid. Example notations: 'org.gradle:gradle-core:2.2', 'org.mockito:mockito-core:1.9.5:javadoc'.
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.
BUILD FAILED in 515ms
Build Failed
2024-09-24 13:33:34,954 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2024-09-24 13:33:35,020 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2024-09-24 13:33:35,021 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '32e49431-3054-4146-9016-6ce5d4b87225', 'installationId': '2133b365-0639-4ab7-af37-5c89e4d8834b', 'sessionId':
'128d9d2b-e096-4fd1-8c8c-2196097d9f90', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.13', 'samcliVersion': '1.121.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '',
'commandName': 'sam build', 'metricSpecificAttributes': {'projectType': 'CFN', 'gitOrigin': None, 'projectName': '22b9f27c36f488a986348fa8d5bfdd71a578a2b90fcea7803f17fc141049b099', 'initialCommit': None},
'duration': 1518, 'exitReason': 'WorkflowFailedError', 'exitCode': 1}}]}
2024-09-24 13:33:35,021 | Unable to find Click Context for getting session_id.
2024-09-24 13:33:35,024 | Sending Telemetry: {'metrics': [{'events': {'requestId': '7dc37973-9b87-4bc7-9bcf-636b3100d990', 'installationId': '2133b365-0639-4ab7-af37-5c89e4d8834b', 'sessionId':
'128d9d2b-e096-4fd1-8c8c-2196097d9f90', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.13', 'samcliVersion': '1.121.0', 'commandName': 'sam build', 'metricSpecificAttributes': {'events':
[{'event_name': 'SamConfigFileExtension', 'event_value': '.toml', 'thread_id': '43449d2fd6aa4a8298339dbdb6cd7e67', 'time_stamp': '2024-09-24 10:33:33.419', 'exception_name': None}, {'event_name':
'SamConfigFileExtension', 'event_value': '.toml', 'thread_id': 'b4d6074e1e2b482e91e2c2ec5dd4eced', 'time_stamp': '2024-09-24 10:33:33.436', 'exception_name': None}, {'event_name': 'BuildFunctionRuntime',
'event_value': 'java21', 'thread_id': '30a5f3de097c4b6483656a68d54f4807', 'time_stamp': '2024-09-24 10:33:33.520', 'exception_name': None}, {'event_name': 'BuildWorkflowUsed', 'event_value': 'java-gradle',
'thread_id': '19f3109f4280404aa35031aa6e4c82ff', 'time_stamp': '2024-09-24 10:33:33.526', 'exception_name': None}]}}}]}
2024-09-24 13:33:35,526 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: JavaGradleWorkflow:GradleBuild - Gradle Failed: FAILURE: Build failed with an exception.
* Where:
Build file '/Users/mantas.gridinas/IdeaProjects/lambda-at-home/build.gradle' line: 17
* What went wrong:
A problem occurred evaluating root project 'lambda-at-home'.
> Supplied String module notation 'shit' is invalid. Example notations: 'org.gradle:gradle-core:2.2', 'org.mockito:mockito-core:1.9.5:javadoc'.
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.
BUILD FAILED in 515ms
2024-09-24 13:33:35,560 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Expected result:
Build is skipped regardless of naming the lambda or not as that is how the aws toolkit idea plugin works
Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
- OS: Mac 14.6
sam --version: 1.121- AWS region: irrelevant
{
"version": "1.121.0",
"system": {
"python": "3.8.13",
"os": "macOS-14.6.1-arm64-arm-64bit"
},
"additional_dependencies": {
"docker_engine": "26.0.0",
"aws_cdk": "Not available",
"terraform": "1.8.3"
},
"available_beta_feature_env_vars": [
"SAM_CLI_BETA_FEATURES",
"SAM_CLI_BETA_BUILD_PERFORMANCE",
"SAM_CLI_BETA_TERRAFORM_SUPPORT",
"SAM_CLI_BETA_RUST_CARGO_LAMBDA"
]
}
lucashuy commented
This kind of sounds like an edge case we did just never considered: ie. what happens if we skip the build, but also pass in the function explicitly to build.
I just wanted to ask about your use case, since sam build will still create an output template with other resources when specifying an explicit resource to build. It just won't build the other resources.
cj-mantas-gridinas commented
My usecase is to run lambdas via intellij with aws toolkit plugin. The plugin runs 2 commands sequentially - build, with explicit lambda name, and local invoke with output template and other resources that would be produced by the build command. The plugin isn't something made by community either. It's provided by AWS (verified, too).
https://plugins.jetbrains.com/plugin/11349-aws-toolkit
The original problem stems from SAM trying to guess what build tool is being used, trying to invoke it, and expecting a partcular file to be present after the build. In the provided case, SAM would expect the result to be a fat jar, instead of a zip of jars, which is a valid deployment in regular lambdas. Just moving the zip file to ARTIFACT_DIR (in makefile configuration) isn't enough, as you must extract it.