Confusing error message when several lambdas reuse same package
benjamin9fin opened this issue · 6 comments
Description
When using a shared codebase for multiple lambdas, it's easy to run into this issue mentioned in the docs:
Hash of zip-archive created with the same content of the files is always identical which prevents unnecessary force-updates of the Lambda resources unless content modifies. If you need to have different filenames for the same content you can specify extra string argument hash_extra.
When calling this module multiple times in one execution to create packages with the same source_path, zip-archives will be corrupted due to concurrent writes into the same file. There are two solutions - set different values for hash_extra to create different archives, or create package once outside (using this module) and then pass local_existing_package argument to create other Lambda resources.
Unfortunately the error message you get in this case does not point to this issue at all:
Traceback (most recent call last):
File "/app/terraform-tests/.terraform/modules/func_summarize/package.py", line 1628, in <module>
main()
File "/app/terraform-tests/.terraform/modules/func_summarize/package.py", line 1624, in main
exit(args.command(args))
File "/app/terraform-tests/.terraform/modules/func_summarize/package.py", line 1514, in build_command
with ZipWriteStream(filename) as zs:
File "/app/terraform-tests/.terraform/modules/func_summarize/package.py", line 316, in __exit__
self.close(failed=True)
File "/app/terraform-tests/.terraform/modules/func_summarize/package.py", line 306, in close
os.unlink(self._tmp_filename)
FileNotFoundError: [Errno 2] No such file or directory:
'builds/eec77390ebbc12d375659820fc1e772251ab42f6370dd22baa4a4d1009a07ec1.zip.tmp'
In my case, this seems to be because I have ~8 lambdas each with identical zip archives due to shared codebase. The writes to the zip archives are corrupting each other because they all have the same filename due to identical content hash.
There's no way to really know this from the error message though. You have to find that section in the docs to get the explanation.
What would be great is if the error reporting could be improved in this case. If we could detect and mention this known issue here in the error message then it could potentially save hours of debugging.
- ✋ I have searched the open/closed issues and my issue is not listed.
I can't provide example code in this case but it's easy to reproduce just by creating several lambdas that have the same source_path.
Good point on updating the error message. I don't have time for that, but maybe someone else can do it.
i got same error when you use for_each for the lambda module
This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days
This issue was automatically closed because of stale in 10 days
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.