Create example infrastructure with LocalStack

[seanttaylor]

• S3 Bucket
• Lambda for hello_world "pipe"
• Cloudwatch log for Lambda

[seanttaylor]

Creating the aws_lambda_event_source_mapping resource on LocalStack was successful (i.e. LocalStack reported no errors). hello_world lambda does not appear to be triggering when new items are introduced to the Kafka stream.

The aws_cloudwatch_log_group resource is created for hello_world lambda on LocalStack successfully but when verifying the log creation using the AWS CLI no logs are indicated. 🤔 It's possible the AWS CLI is not configured correctly with LocalStack. Exploring this next.

[seanttaylor]

aws --endpoint-url=http://localhost:4566 logs tail /aws/lambda/hello_world --follow throws an error:

An error occurred (ResourceNotFoundException) when calling the FilterLogEvents operation: The specified log group does not exist

☝️ This error happened because the log group name was wrong:
aws --endpoint-url=http://localhost:4566 logs tail /aws/lambda/hello_world --followvs aws --endpoint-url=http://localhost:4566 logs tail /aws/lambda/hello-world --follow note the hyphen vs. underscore in hello-world

[seanttaylor]

That said: with the correct log group there is no log data for the hello-world lambda, which suggests the function isn't being invoked at all. Next exploration will focus on this.

[seanttaylor]

Doing aws --endpoint-url=http://localhost:4566 lambda invoke --function-name hello-world out --log-type Tail returns the following:

{
    "StatusCode": 200,
    "FunctionError": "Unhandled",
    "LogResult": "",
    "ExecutedVersion": "$LATEST"
}

Is the hello-world lambda throwing some error? 🤔

[seanttaylor]

Turns out reviewing the docker-compose logs for the localstack_main container provided some more clues. There are some log items of interest namely:

docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=nodejs16.x&fromImage=lambci%2Flambda: Not Found ("manifest for lambci/lambda:nodejs16.x not found: manifest unknown: manifest unknown")

docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.41/containers/create: Not Found ("No such image: lambci/lambda:nodejs16.x")

Which suggests the Localstack Lambda service cannot create the container for the Lambda. Next exploration.

Further: if the localstack_main service LAMDBA_EXECUTOR environment variable is changed from its default (i.e. docker) to local a different error suggesting Localstack cannot execute lambdas that are ES6 modules appears. Next exploration.

See issue: localstack/localstack#6021

[seanttaylor]

Per the Localstack issue referenced above ☝️ using CommonJS modules allowed the Lambda to execute without errors. Log output was available by running the aws logs as expected. Note the LAMDBA_EXECUTOR environment variable was set to local in the docker-compose configuration as well.