terraform-aws-modules/terraform-aws-lambda

Using build_in_docker running docker in docker

reikje opened this issue · 4 comments

reikje commented

Description

I am struggling using the build_in_docker option when running inside a VSCode devcontainer.

I arrived here because my Lambda needs a Python dependency called cryptography and I am hitting /lib64/libc.so.6: version 'GLIBC_2.28' not found when I run terraform apply on my Macbook.

My project is running inside a .devcontainer and I am mounting "source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind" into that container. When I then run terraform apply inside the container, it basically is running docker in docker utilising Docker Desktop on the host. Using the build_in_docker option will result in the following error:

21826ea7c95e: Pull complete
│ Digest: sha256:79efc767616c002acf83758a24da5fb02d3d079313321b0de304bbfec402b77d
│ Status: Downloaded newer image for public.ecr.aws/sam/build-python3.10:latest
│ docker: Error response from daemon: Mounts denied: 
│ The path /root/.ssh/known_hosts is not shared from the host and is not known to Docker.

Here is the mount causing the issue. I am really not a docker expert but I think instead of using /root (which is $HOME inside my devcontainer) I need to use the path actual available on the host. There seem to be no option to set the folder for the .ssh directory ...

  • [ x ] ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

  1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
  2. Re-initialize the project root to pull down modules: terraform init
  3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

  • Module version [Required]:
    6.0.1
  • Terraform version:
    1.6.1
  • Provider version(s):
    5.21.0

Reproduction Code [Required]

.devcontainer/devcontainer.json:

{
  "name": "Python 3",
  "build": {
    "dockerfile": "Dockerfile",
    "context": "..",
    "args": {
    }
  },
  "customizations": {
    "vscode": {
      "settings": {
        "editor.suggest.insertMode": "replace",
        "python.testing.pytestArgs": [
          "tests"
        ],
        "python.testing.unittestEnabled": false,
        "python.testing.pytestEnabled": true,
        "python.defaultInterpreterPath": "/var/folders/.venv/bin/python",
        "python.analysis.indexing": true,
        "python.analysis.packageIndexDepths": [
            { "name": "", "depth": 2, "includeAllSymbols": true }
        ]
      },

      "extensions": [
        "ms-python.python",
        "charliermarsh.ruff",
        "hashicorp.terraform",
        "njpwerner.autodocstring"
      ]
    }
  },
  "remoteUser": "root",
  "runArgs": ["--add-host=host.docker.internal:host-gateway"],
  "containerEnv": {
    "GITHUB_WORKSPACE": "${localWorkspaceFolder}"
  },
  "mounts": [
      "source=${env:HOME}${env:USERPROFILE}/.aws,target=/root/.aws,type=bind,readonly",
      "source=${env:HOME}${env:USERPROFILE}/.ssh,target=/root/.ssh,type=bind,readonly",
      "source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
  ],
  "features": {
    "github-cli": "latest"
  },
  "workspaceMount": "source=${localWorkspaceFolder},target=/var/folders,type=bind",
  "workspaceFolder": "/var/folders"
}

lambda.tf:

module "lambda_function" {
  source              = "terraform-aws-modules/lambda/aws"
  version             = "~> 6.0"
  timeout             = 300
  source_path         = "../../src"
  function_name       = "whatever"
  handler             = "app.handler"
  runtime             = "python3.10"
  create_sam_metadata = true
  publish             = true
  build_in_docker     = true

  allowed_triggers = {
    AnyRule = {
      principal  = "events.amazonaws.com"
      source_arn = "arn:aws:events:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:rule/*"
    }
  }

  tags = local.tags
}

Steps to reproduce the behavior:

  • create a VSCode devcontainer
  • open the project in the devcontainer
  • run terraform apply in a module that has the build_in_docker option set to true

Expected behavior

The build runs properly inside a docker (in docker) container.

Actual behavior

The build fails with:

...
│ 21826ea7c95e: Pull complete
│ Digest: sha256:79efc767616c002acf83758a24da5fb02d3d079313321b0de304bbfec402b77d
│ Status: Downloaded newer image for public.ecr.aws/sam/build-python3.10:latest
│ docker: Error response from daemon: Mounts denied: 
│ The path /root/.ssh/known_hosts is not shared from the host and is not known to Docker.

.. unless the host system has a /root/.ssh folder, which is not the case in MacOS.

villekr commented

I'm also looking at the similar case i.e. trying to use terraform-aws-lambda -module within dockerized build container.

I think mounting .ssh/known_hosts is necessary only if accessing packages in private repos via ssh. So basically it's possibly to use local version of terraform-aws-lambda-repository and modify package.py docker_run_command-function a bit. I commented out this part as a temp workaround. And of course one need to refer then to local path of terraform-aws-lambda directory.

    # docker_cmd.extend([
    #     # '-v', '{}/.ssh/id_rsa:/root/.ssh/id_rsa:z'.format(home),
    #     '-v', '{}/.ssh/known_hosts:/root/.ssh/known_hosts:z'.format(home),
    # ])

But then I'm facing another issue when install npm packages:

Error: local-exec provisioner error
│ 
│   with module.nodejs_lambda_layer.null_resource.archive[0],
│   on modules/terraform-aws-lambda/package.tf line 67, in resource "null_resource" "archive":
│   67:   provisioner "local-exec" {
│ 
│ Error running command
│ './builds/lambda_layer/9b33c113e714cbd7294f9da60e90d6c4ca1f8babd889b16ef17adc9f073a6fe6.plan.json':
│ exit status 1. Output: zip: creating
│ './builds/lambda_layer/9b33c113e714cbd7294f9da60e90d6c4ca1f8babd889b16ef17adc9f073a6fe6.zip'
│ archive
│ > docker images '--format={{.ID}}' ecs-test-pipeline-infra-nodejs18.x-build
│ Installing npm requirements: ./../nodejs-lambda/package.json
│ > mktemp -d terraform-aws-lambda-XXXXXXXX #
│ /tmp/terraform-aws-lambda-ljdocj4k
│ > cd /tmp/terraform-aws-lambda-ljdocj4k
│ > docker run --rm -w /var/task -v
│ /tmp/terraform-aws-lambda-ljdocj4k:/var/task:z --entrypoint '' 8a0a80350522
│ /bin/sh -c 'npm install && chown -R 0:0 .'
│ npm ERR! code ENOENT
│ npm ERR! syscall open
│ npm ERR! path /var/task/package.json
│ npm ERR! errno -2
│ npm ERR! enoent ENOENT: no such file or directory, open
│ '/var/task/package.json'
│ npm ERR! enoent This is related to npm not being able to find a file.
│ npm ERR! enoent
github-actions commented

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

github-actions commented

This issue was automatically closed because of stale in 10 days