Could not locate source_path

Question

Could not locate source_path

emre141 opened this issue 3 years ago · 15 comments

emre141 commented 3 years ago

Description

Terraform plan failed due to Could not locate source_path regarding to use "${path.module}/sourcecode"

⚠️ Note

Before you submit an issue, please perform the following first:

Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
Re-initialize the project root to pull down modules: terraform init
Re-attempt your terraform plan or apply and check if the issue still persists

Versions

Terraform:
Terraform v0.14.3

provider registry.terraform.io/hashicorp/archive v2.2.0
provider registry.terraform.io/hashicorp/aws v3.62.0
provider registry.terraform.io/hashicorp/external v2.1.0
provider registry.terraform.io/hashicorp/local v2.1.0
provider registry.terraform.io/hashicorp/null v3.1.0
provider registry.terraform.io/hashicorp/random v3.1.0

Provider(s):
provider registry.terraform.io/hashicorp/aws v3.62.0
Module:
Downloading terraform-aws-modules/lambda/aws 2.20.0 for lambda_function...
lambda_function in .terraform/modules/lambda_function
Downloading terraform-aws-modules/lambda/aws 2.20.0 for lambda_layer_local...
lambda_layer_local in .terraform/modules/lambda_layer_local
Downloading terraform-aws-modules/cloudwatch/aws 2.1.0 for log_metric_filter...
log_metric_filter in .terraform/modules/log_metric_filter/modules/log-metric-filter
Downloading terraform-aws-modules/lambda/aws 2.20.0 for package_dir...

Reproduction

Steps to reproduce the behavior:

No

Yes

Code Snippet to Reproduce

terragrunt plan

Expected behavior

I expected to success plan output

Actual behavior

[terragrunt] 2021/10/19 22:25:07 Running command: terraform plan -lock-timeout=20m -parallelism=40
Acquiring state lock. This may take a few moments...
module.package_file_with_pip_requirements.null_resource.archive[0]: Refreshing state... [id=2827457759683703423]
random_pet.this: Refreshing state... [id=able-jackal]
module.package_file_with_pip_requirements.local_file.archive_plan[0]: Refreshing state... [id=4b0210d0d55788bc1bb4bdbd3ece4947e01b3932]
module.package_dir.local_file.archive_plan[0]: Refreshing state... [id=659c26b6176a1e6731e90d5af8f76bb742e9f0ea]
module.package_dir.null_resource.archive[0]: Refreshing state... [id=3477677468785454566]
aws_cloudwatch_event_rule.trigger_lambda_function: Refreshing state... [id=funcChromeFeed-rule]
module.lambda_function.aws_cloudwatch_log_group.lambda[0]: Refreshing state... [id=/aws/lambda/funcChromeFeed-test]
module.lambda_function.aws_iam_role.lambda[0]: Refreshing state... [id=funcChromeFeed-test]
module.log_metric_filter.aws_cloudwatch_log_metric_filter.this[0]: Refreshing state... [id=error-metric-/aws/lambda/funcChromeFeed-test]
module.lambda_function.aws_iam_policy.logs[0]: Refreshing state... [id=arn:aws:iam::681105386544:policy/funcChromeFeed-test-logs]
module.lambda_function.aws_iam_role_policy_attachment.logs[0]: Refreshing state... [id=funcChromeFeed-test-20211012202019408900000001]

Error: failed to execute "python3": Could not locate source_path "/Users/emregundogdu/Desktop/onedio/scripts/reporting_scripts/code/seo_Chrome_Feed/builds/b41fcb4f1bc02b69b8acd0661bf2f4c7890d25b76a01a73ad75ee9d91fe1c372.zip". Paths are relative to directory where terraform plan is being run ("/Users/emregundogdu/Desktop/onedio/scripts/reporting_scripts/code/seo_Chrome_Feed")

Terminal Output Screenshot(s)

Answer 1 · 2021-10-20T07:49:33.000Z

Make sure that you specify the correct source_path. The error is "Paths are relative to directory where terraform plan is being run".

You can also see this comment - #27 (comment)

Answer 2 · 2021-10-20T07:59:12.000Z

Make sure that you specify the correct source_path. The error is "Paths are relative to directory where terraform plan is being run".

You can also see this comment - #27 (comment)

Yes I am sure, the module called as following line,

module "package_dir" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "2.20.0"
  create_function = false
  source_path = "${path.module}/sourcecode"
  runtime     = "python3.8"
}

As well my module folder tree as below

├── artifactdir
│   └── b41fcb4f1bc02b69b8acd0661bf2f4c7890d25b76a01a73ad75ee9d91fe1c372.plan.json
├── backend.tf
├── build
│   └── funcChromeFeed-dev-hello-test.zip
├── builds
│   └── 089b27bc270053fb9e310f0c903c4c025641b00d62f94cd4290213964bc6faae.plan.json
├── main.tf
├── output.tf
├── sourcecode
│   ├── client_secrets.p12
│   ├── constants.pickle
│   ├── ga_get_data.py
│   ├── handler.py
│   ├── requirements.txt
│   ├── seo_google_chrome_feed.py
│   └── slackbot_functions.py
├── terragrunt.hcl
├── variables.tf
└── version.tf

I mean there is a difference only i was using terragrunt instead of directly using terraform, is it important what i am using ?

Answer 3 · 2021-10-20T08:13:15.000Z

I don't see an immediate problem with this code and structure, but I don't understand why artifactdir directory is used and not builds (have you changed it in artifacts_dir variable?)

If you run terragrunt plan, you may need to wrap a value of source_path in jsonencode(). There is a bug in terragrunt.

PS: Just discovered your edit about using terragrunt.

Answer 4 · 2021-10-20T08:23:09.000Z

Yeah I just tried lots of different alternative that why are you see the artifact dir on there, So you mean if have to use source_path in jsonencode() , I would try to use source_path = jsonencode("${path.module}/sourcecode"), is it right? If it is not right , could you share complete line of the source_path in jsonencode()

Answer 5 · 2021-10-20T08:55:34.000Z

@antonbabenko I have got similar error over again once I update source_path as following module call,

module "package_dir" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "2.20.0"
  create_function = false
  source_path = jsonencode("${path.module}/sourcecode")
  runtime     = "python3.8"
}

terragrunt plan

Error: failed to execute "python3": Could not locate source_path ""./sourcecode"".  Paths are relative to directory where `terraform plan` is being run ("/Users/emregundogdu/Desktop/onedio/scripts/reporting_scripts/code/seo_Chrome_Feed")

Folder tree

.
├── backend.tf
├── build
│   └── funcChromeFeed-dev-hello-test.zip
├── main.tf
├── output.tf
├── sourcecode
│   ├── client_secrets.p12
│   ├── constants.pickle
│   ├── ga_get_data.py
│   ├── handler.py
│   ├── requirements.txt
│   ├── seo_google_chrome_feed.py
│   └── slackbot_functions.py
├── terragrunt.hcl
├── variables.tf
└── version.tf

Answer 6 · 2021-10-20T08:58:37.000Z

Here is my real terragrunt.hcl config:

inputs = {
  source_path   = jsonencode("${abspath("${get_terragrunt_dir()}/../../backend")}/function1")
  artifacts_dir = abspath("${get_terragrunt_dir()}/../builds")
}

I don't think it makes a lot of sense to mix path.module when using Terragrunt.

Answer 7 · 2021-10-20T10:13:46.000Z

@antonbabenko
I have got same error regarding the change input variables that described above, I guess there is mistake at the source_path or artifacts_dir I would like to share my directory tree again that you may suggest the absolute path after ${get_terragrunt_dir()}

Terragrunt hcl file that that lives in the working directory

include {
  path = find_in_parent_folders()
}
inputs = {
  source_path   = jsonencode("${abspath("${get_terragrunt_dir()}")}/sourcecode")
  artifacts_dir = abspath("${get_terragrunt_dir()}/builds")
}

Module definition as below

module "package_dir" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "2.20.0"
  create_function = false
  source_path = var.source_path
  runtime     = "python3.8"
  artifacts_dir = var.artifacts_dir
}

Working directory tree

.
├── backend.tf
├── build
│   └── funcChromeFeed-dev-hello-test.zip
├── main.tf
├── output.tf
├── sourcecode
│   ├── client_secrets.p12
│   ├── constants.pickle
│   ├── ga_get_data.py
│   ├── handler.py
│   ├── requirements.txt
│   ├── seo_google_chrome_feed.py
│   └── slackbot_functions.py
├── terragrunt.hcl
├── variables.tf
└── version.tf

Error Message

Error: failed to execute "python3": Could not locate source_path ""/Users/emregundogdu/Desktop/onedio/scripts/reporting_scripts/code/seo_Chrome_Feed/sourcecode"".  Paths are relative to directory where `terraform plan` is being run ("/Users/emregundogdu/Desktop/onedio/scripts/reporting_scripts/code/seo_Chrome_Feed")

Answer 8 · 2021-10-20T10:28:26.000Z

I am a bit lost in the way you use terragrunt and terraform together. Why do you have module block and terragrunt.hcl?

Here is my complete terragrunt.hcl and I don't have any tf-files near it:

terraform {
  source = "tfr:///terraform-aws-modules/lambda/aws?version=2.20.0"
}

include {
  path = find_in_parent_folders()
}

inputs = {
  function_name = "function1"
  handler       = "webhook_handler.webhook_handler"
  runtime       = "python3.8"
  timeout       = 30
  memory_size   = 512

  publish = true

  // terragrunt bug: use jsonencode when "type = any"
  source_path = jsonencode("${abspath("${get_terragrunt_dir()}/../../backend")}/function1")

  artifacts_dir = abspath("${get_terragrunt_dir()}/../builds")
}

Try putting fixed string value into source_path into your code to avoid using variables and HCL/terragrunt functions to begin with.

Answer 9 · 2021-10-20T11:26:44.000Z

I got it but I have to use terragrunt and terraform together there a few terraform modules that are depend lambda onces, but So it is possible to use together at a time using input block and variables.tf in the tf file

What I did not understood from here the full path of the lambda source code that will execute python , Could you share the directory tree from terragrunt.hcl live ? I have shared my directory tree to pointing lambda source code as sourcecode directory so I would like to compare the source path input parameter as you shared your onces source_path = jsonencode("${abspath("${get_terragrunt_dir()}/../../backend")}/function1") , my source code directory sourcecode that is living same level of terragrunt.hcl file so what should i use the right path within following value jsonencode("${abspath("${get_terragrunt_dir()}/path_to_my_source_code")}/function1")

Answer 10 · 2021-10-20T11:32:30.000Z

My tree is very simple:

.
├── function1
│   └── terragrunt.hcl
├ ... # other unrelated dirs with `terragrunt.hcl` files

The backend/function1 directory with the source code is several levels outside of the place I run terragrunt command.

Answer 11 · 2021-10-20T12:04:01.000Z

I didn't understand enough could you refactor source_path = jsonencode("${abspath("${get_terragrunt_dir()}/../../backend")}/function1") as my directory tree onces for me ?

My Directory Tree

.
├── backend.tf
├── build
│   └── funcChromeFeed-dev-hello-test.zip
├── main.tf
├── output.tf
├── sourcecode
│   ├── client_secrets.p12
│   ├── constants.pickle
│   ├── ga_get_data.py
│   ├── handler.py
│   ├── requirements.txt
│   ├── seo_google_chrome_feed.py
│   └── slackbot_functions.py
├── terragrunt.hcl
├── variables.tf
└── version.tf

Answer 12 · 2021-10-20T13:17:48.000Z

Try with simple string value first:

source_path = jsonencode("/Users/emregundogdu/Desktop/onedio/scripts/reporting_scripts/code/seo_Chrome_Feed/sourcecode")

If it does not, try to move sourcecode folder outside (one level up, for eg).

Answer 13 · 2022-01-11T01:03:08.000Z

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

Answer 14 · 2022-01-22T00:55:37.000Z

This issue was automatically closed because of stale in 10 days

Answer 15 · 2022-11-09T02:38:16.000Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.