deploy-tre script fails with error "Error: GoModulesBuilder:Build - Builder Failed: stat /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/terraform_open_source_parameter_parser/terraform_open_source_parameter_parser: directory not found"

Question

deploy-tre script fails with error "Error: GoModulesBuilder:Build - Builder Failed: stat /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/terraform_open_source_parameter_parser/terraform_open_source_parameter_parser: directory not found"

kaul-abhi opened this issue a year ago · 2 comments

Hello Team,
When i am running script for deployment and when command "sam build" runs it fails with error "Error: GoModulesBuilder:Build - Builder Failed: stat /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/terraform_open_source_parameter_parser/terraform_open_source_parameter_parser: directory not found".

Go version : go version go1.20.7
No changes is made in template.yaml for CodeUri section but still it fails.
PFB the error logs:

$ sam --version
SAM CLI, version 1.100.0
$ aws --version
aws-cli/2.13.20 Python/3.11.5 Linux/6.0.7-301.fc37.x86_64 exe/x86_64.alpine.3 prompt/off
$ python3.9 -V
Python 3.9.13
$ pip3 --version
pip 23.3.1 from /usr/lib/python3.11/site-packages/pip (python 3.11)
$ export PATH=$PATH:/usr/local/go/bin
$ which go
/usr/local/go/bin/go
$ go version
go version go1.20.7 linux/amd64
$ /pipeline/treinstall
$TRE_REPO_LOCATION = 'https://github.com/aws-samples/service-catalog-engine-for-terraform-os.git'
AWS_REGION : eu-west-1
$AWS_REGION = eu-west-1
Clone the Terraform Reference Engine code from github.
Cloning into 'service-catalog-engine-for-terraform-os'...
10:52:19 [INFO] [treinstall] cleaning template.yml file so to copy customized template.yaml
copying customized template.yml file into TRE directory
10:52:19 [INFO] [treinstall] current profile information for troubleshooting in case of publish errors
{
"UserId": "AROAVRDUU2NP36G2CFAUV:GitLabRunner-327317-145418590",
"Account": "380349240159",
"Arn": "arn:aws:sts::380349240159:assumed-role/csc-runner-default/GitLabRunner-327317-145418590"
}
running the deploy-tre script
AWS account: 380349240159
AWS region: eu-west-1
Creating a Python virtual environment
Building the ServiceCatalogTerraformOSParameterParser function
go: creating new go.mod: module terraform_open_source_parameter_parser
go: to add module requirements and sums:
go mod tidy
go: finding module for package github.com/aws/aws-sdk-go/aws
go: finding module for package github.com/aws/aws-sdk-go/aws/arn
go: finding module for package github.com/stretchr/testify/mock
go: finding module for package github.com/aws/aws-lambda-go/lambda
go: finding module for package github.com/aws/aws-sdk-go/aws/credentials
go: finding module for package github.com/aws/aws-sdk-go/aws/credentials/stscreds
go: finding module for package github.com/aws/aws-sdk-go/aws/session
go: finding module for package github.com/aws/aws-sdk-go/service/s3
go: finding module for package github.com/aws/aws-sdk-go/service/s3/s3manager
go: finding module for package github.com/hashicorp/hcl/v2/hclparse
go: finding module for package github.com/hashicorp/terraform-config-inspect/tfconfig
go: downloading github.com/stretchr/testify v1.8.4
go: downloading github.com/hashicorp/terraform-config-inspect v0.0.0-20230925220900-5a6f8d18746d
go: downloading github.com/aws/aws-lambda-go v1.41.0
go: downloading github.com/hashicorp/hcl/v2 v2.19.1
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading github.com/aws/aws-sdk-go v1.47.2
go: found github.com/aws/aws-lambda-go/lambda in github.com/aws/aws-lambda-go v1.41.0
go: found github.com/aws/aws-sdk-go/aws in github.com/aws/aws-sdk-go v1.47.2
go: found github.com/aws/aws-sdk-go/aws/arn in github.com/aws/aws-sdk-go v1.47.2
go: found github.com/aws/aws-sdk-go/aws/credentials in github.com/aws/aws-sdk-go v1.47.2
go: found github.com/aws/aws-sdk-go/aws/credentials/stscreds in github.com/aws/aws-sdk-go v1.47.2
go: found github.com/aws/aws-sdk-go/aws/session in github.com/aws/aws-sdk-go v1.47.2
go: found github.com/aws/aws-sdk-go/service/s3 in github.com/aws/aws-sdk-go v1.47.2
go: found github.com/aws/aws-sdk-go/service/s3/s3manager in github.com/aws/aws-sdk-go v1.47.2
go: found github.com/hashicorp/hcl/v2/hclparse in github.com/hashicorp/hcl/v2 v2.19.1
go: found github.com/hashicorp/terraform-config-inspect/tfconfig in github.com/hashicorp/terraform-config-inspect v0.0.0-20230925220900-5a6f8d18746d
go: found github.com/stretchr/testify/mock in github.com/stretchr/testify v1.8.4
go: downloading github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f
go: downloading github.com/zclconf/go-cty v1.13.0
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading github.com/go-test/deep v1.0.3
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading github.com/google/go-cmp v0.3.1
go: downloading github.com/stretchr/objx v0.5.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/jmespath/go-jmespath v0.4.0
go: downloading github.com/agext/levenshtein v1.2.2
go: downloading github.com/apparentlymart/go-textseg/v15 v15.0.0
go: downloading github.com/mitchellh/go-wordwrap v1.0.0
go: downloading github.com/kr/pretty v0.1.0
go: downloading github.com/kylelemons/godebug v1.1.0
go: downloading github.com/apparentlymart/go-textseg v1.0.0
go: downloading golang.org/x/net v0.17.0
go: downloading github.com/sergi/go-diff v1.0.0
go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0
go: downloading golang.org/x/text v0.13.0
go: downloading github.com/kr/text v0.1.0
go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1
go: downloading gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
go: downloading gopkg.in/yaml.v2 v2.2.8
Building the Lambda code
Collecting wheel
Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/fa/7f/4c07234086edbce4a0a446209dc0cb08a19bb206a3ea53b2f56a403f983b/wheel-0.41.3-py3-none-any.whl.metadata
Downloading wheel-0.41.3-py3-none-any.whl.metadata (2.2 kB)
Downloading wheel-0.41.3-py3-none-any.whl (65 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.8/65.8 kB 674.9 kB/s eta 0:00:00
Installing collected packages: wheel
Successfully installed wheel-0.41.3
[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
Collecting boto3>=1.26.105 (from -r lambda-functions/state_machine_lambdas/requirements.txt (line 1))
Obtaining dependency information for boto3>=1.26.105 from https://files.pythonhosted.org/packages/72/01/eded6bd8244670f1609743a0e63e097d86d8f79801d323fc18b00267cf20/boto3-1.28.77-py3-none-any.whl.metadata
Downloading boto3-1.28.77-py3-none-any.whl.metadata (6.7 kB)
Collecting botocore>=1.29.105 (from -r lambda-functions/state_machine_lambdas/requirements.txt (line 2))
Obtaining dependency information for botocore>=1.29.105 from https://files.pythonhosted.org/packages/56/05/f8d8d52d15e713909facb6a13276822fea04da16e3d4de08c8860c9884c7/botocore-1.31.77-py3-none-any.whl.metadata
Downloading botocore-1.31.77-py3-none-any.whl.metadata (6.1 kB)
Collecting jmespath<2.0.0,>=0.7.1 (from boto3>=1.26.105->-r lambda-functions/state_machine_lambdas/requirements.txt (line 1))
Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting s3transfer<0.8.0,>=0.7.0 (from boto3>=1.26.105->-r lambda-functions/state_machine_lambdas/requirements.txt (line 1))
Obtaining dependency information for s3transfer<0.8.0,>=0.7.0 from https://files.pythonhosted.org/packages/5a/4b/fec9ce18f8874a96c5061[42](https://code.siemens.com/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/-/jobs/145418590#L42)2625ba86c3ee1e6587ccd92ff9f5bf7bd91b2/s3transfer-0.7.0-py3-none-any.whl.metadata
Downloading s3transfer-0.7.0-py3-none-any.whl.metadata (1.8 kB)
Collecting python-dateutil<3.0.0,>=2.1 (from botocore>=1.29.105->-r lambda-functions/state_machine_lambdas/requirements.txt (line 2))
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 4.9 MB/s eta 0:00:00
Collecting urllib3<2.1,>=1.25.4 (from botocore>=1.29.105->-r lambda-functions/state_machine_lambdas/requirements.txt (line 2))
Obtaining dependency information for urllib3<2.1,>=1.25.4 from https://files.pythonhosted.org/packages/d2/b2/b15785[51](https://code.siemens.com/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/-/jobs/145418590#L51)92a68541a91ba7b2bbcb91f1b4faa51f8bae38d8005c034be[52](https://code.siemens.com/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/-/jobs/145418590#L52)4/urllib3-2.0.7-py3-none-any.whl.metadata
Downloading urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore>=1.29.105->-r lambda-functions/state_machine_lambdas/requirements.txt (line 2))
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Downloading boto3-1.28.77-py3-none-any.whl (135 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 135.8/135.8 kB 9.1 MB/s eta 0:00:00
Downloading botocore-1.31.77-py3-none-any.whl (11.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.3/11.3 MB 36.7 MB/s eta 0:00:00
Downloading s3transfer-0.7.0-py3-none-any.whl (79 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.8/79.8 kB 8.6 MB/s eta 0:00:00
Downloading urllib3-2.0.7-py3-none-any.whl (124 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 kB 23.1 MB/s eta 0:00:00
Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3
Successfully installed boto3-1.28.77 botocore-1.31.77 jmespath-1.0.1 python-dateutil-2.8.2 s3transfer-0.7.0 six-1.16.0 urllib3-2.0.7
[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
SAM CLI now collects telemetry to better understand customer needs.
You can OPT OUT and disable telemetry collection by setting the
environment variable SAM_CLI_TELEMETRY=0 in your shell.
Thanks for your help!
Learn More: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-telemetry.html
Building codeuri: /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/provisioning-operations-handler runtime: python3.9 metadata: {} architecture: x86_64 functions: TerraformEngineProvisioningHandlerLambda, TerraformEngineTerminateHandlerLambda
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
Building codeuri: /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/terraform_open_source_parameter_parser runtime: go1.x metadata: {} architecture: x86_64 functions: ParameterParser, ExternalParameterParser
Running GoModulesBuilder:Build
Build Failed
Error: GoModulesBuilder:Build - Builder Failed: stat /builds/horizon/platform-verticals/reference/aws-service-catalog-sample-pipeline/service-catalog-engine-for-terraform-os/lambda-functions/terraform_open_source_parameter_parser/terraform_open_source_parameter_parser: directory not found
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1

KINDLY HELP.

Answer 1 · 2023-11-08T07:18:09.000Z

base docker image is of alpine and packages installed are : bash
py-pip
jq
figlet
go
wget
unzip
git
gawk
build-base
python3
py3-pip
zlib-dev
readline-dev
ncurses-dev
openssl-dev
sqlite-dev
tk-dev
gdbm-dev
libc-dev
bzip2
# Required for testing with go
gcompat
libc-dev
zip
gcc
# Required for information layer serverless apps local development
make
openssl
# Used with aws-cli
curl
groff \

Answer 2 · 2023-11-08T09:37:36.000Z

@zillidan @smaly-amazon this error had come before also. #27. Please help