CloudWatch LogGroups Not Getting Created

[selected-pixel-jameson]

I have an expressjs App that is setup to run from within a AWS Lambda function. When I deploy this app to my development environment the lambda console logs show up, but it doesn't create a new CloudWatch LogGroup as specified in the configuration.

If I run the lambda function locally and generate logs it will create a CloudWatch Log Group for the local environment.

The Lambda Functions are connecting to an RDS instance so they are contained within a VPC.

The Lambda has been assigned the CloudWatchFullAccess policy so it should not be a permissions error.

I've looked at the Lambda logs and I'm not seeing any errors coming through related to this.

const env = process.env.NODE_ENV || 'local'
const config = require('../../config/env.json')[env]
const winston = require('winston')
const WinstonCloudwatch = require('winston-cloudwatch')
const crypto = require('crypto')

let startTime = new Date().toISOString()
const logger = winston.createLogger({
  exitOnError: false,
  level: 'info',
  transports: [
    new winston.transports.Console({
      json: true,
      colorize: true,
      level: 'info'
    }),
    new WinstonCloudwatch({
      awsAccessKeyId: config.aws.accessKeyId,
      awsSecretKey: config.aws.secretAccessKey,
      logGroupName: 'my-api-' + env,
      logStreamName: function () {
        // Spread log streams across dates as the server stays up
        let date = new Date().toISOString().split('T')[0]
        return 'my-requests-' + date + '-' +
          crypto.createHash('md5')
            .update(startTime)
            .digest('hex')
      },
      awsRegion: 'us-east-1',
      jsonMessage: true
    })
  ]
})

const winstonStream = {
  write: (message, encoding) => {
    // use the 'info' log level so the output will be picked up by both transports
    logger.info(message)
  }
}

module.exports.logger = logger
module.exports.winstonStream = winstonStream

Then within my express app.

const morgan = require('morgan')
const { winstonStream } = require('./providers/loggers')
app.use(morgan('combined', { stream: winstonStream }

[lazywithclass]

What version are you using?

Also to help in debugging the problem reduce your configuration to the simplest form possible, so for example remove express and just run the bit that sends logs to AWS.

[selected-pixel-jameson]

I'm running the following versions.

    "winston": "^3.2.1",
    "winston-cloudwatch": "^2.0.9"

I'm not sure how I would remove express and still test the logging? The logging works when I run express locally so I'm assuming this has something to do with AWS permissions and is probably not even appropriate for this area, but I figured I would see if someone had an idea and figured it would be a good place to have it documented if someone else had the same issue in the future.

[selected-pixel-jameson]

I'm not positive as I haven't implemented it yet on this project, but I believe the issue has to do with the Lambda being in a VPC. When a lambda is in a VPC it loses access to the internet. In order to allow it to have access to the internet you must perform the tasks outlined in this article.

https://aws.amazon.com/premiumsupport/knowledge-center/internet-access-lambda-function/

When I run this locally it has access to the internet and is thus able to create the logs. Hopefully once I implement this it will work correctly.

[lazywithclass]

I suggested removing express to start seeing where the problem is, if the code works outside your deployment environment then you know where the problem is.

So from your second comment I get it you saw logs flowing in outside the VPC, right?

I have very little experience with VPCs, but I remember this project being used successfully in the past by someone that was using Lamba, ktxhbye was added to support a need they had.

[selected-pixel-jameson]

Confirming that the problem was related to the lambda function being in a VPC and not granted public access to the internet through Subnets, Route Tables, NAT and Internet Gateways as described within this post. https://gist.github.com/reggi/dc5f2620b7b4f515e68e46255ac042a7

[lazywithclass]

Great!