Log streaming file for Amazon Linux 2

[opeyemi-paystack]

Hi!

Please can we get an Amazon Linux 2 specific file for streaming logs to cloudwatch? The current file contains commands that do not work on AMI2 environments.

The documentation reflects this here.

[r3na]

Any update? I am facing same issue...
#109

[opeyemi-paystack]

I looked at the differences from the docs and was able to solve my issue with a file with contents similar to this:

###################################################################################################
#### Copyright 2016 Amazon.com, Inc. or its affiliates. All Rights Reserved.
####
#### Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file
#### except in compliance with the License. A copy of the License is located at
####
####     http://aws.amazon.com/apache2.0/
####
#### or in the "license" file accompanying this file. This file is distributed on an "AS IS"
#### BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
#### License for the specific language governing permissions and limitations under the License.
###################################################################################################
###################################################################################################
#### The following file installs and configures the AWS CloudWatch Logs agent to push logs to a Log
#### Group in CloudWatch Logs.
#### 
#### The configuration below sets the logs to be pushed, the Log Group name to push the logs to and
#### the Log Stream name as the instance id. The following files are examples of logs that will be
#### streamed to CloudWatch Logs in near real time:
#### 
#### /var/log/messages
#### /var/log/dmesg 
#### 
#### You can then access the CloudWatch Logs by accessing the AWS CloudWatch Console and clicking
#### the "Logs" link on the left. The Log Group name will follow this format:
####
#### /aws/elasticbeanstalk/<environment name>/<full log name path>
####
#### Please note this configuration can be used additionally to the "Log Streaming" feature:
#### http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.cloudwatchlogs.html
###################################################################################################
packages:
  yum:
    awslogs: []

option_settings:
  - namespace: aws:elasticbeanstalk:cloudwatch:logs
    option_name: StreamLogs
    value: true
  - namespace: aws:elasticbeanstalk:cloudwatch:logs
    option_name: DeleteOnTerminate
    value: false
  - namespace: aws:elasticbeanstalk:cloudwatch:logs
    option_name: RetentionInDays
    value: 3    

files:
  "/etc/awslogs/awscli.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [plugins]
      cwlogs = cwlogs
      [default]
      region = `{"Ref":"AWS::Region"}`

  "/etc/awslogs/config/logs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [/var/log/messages]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/messages"]]}`
      log_stream_name = {instance_id}
      file = /var/log/messages

      [/var/log/dmesg]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/dmesg"]]}`
      log_stream_name = {instance_id}
      file = /var/log/dmesg

commands:
  "01":
    command: systemctl enable awslogsd.service
  "02":
    command: systemctl restart awslogsd

The difference is that AMI2 uses a different start system, so the commands use systemctl instead.

[QuattroViper]

I also struggled with this issue. I can confirm that @opeyemi-paystack's solution works and would like to see an updated file for the AWS Linux v2 AMI.

[dankhen]

Thanks @opeyemi-paystack . I modified the commands in the example per your fix, and kept the legacy version as logs-streamtocloudwatch-linux-alami.config.

I only modified the commands, and didn't make your other changes, as they might be specific to what you need. I wanted to keep the example as close as possible to the original, only adapting for the AL2 OS changes.

[aconchillo]

I'm a bit confused about this one. I'm using Amazon Linux 2 with EB Docker 3.0.3 platform and it doesn't use the old awslogsd agent but amazon-cloudwatch-agent. Logs stream fine except for the Docker logs (which is basically what we want): https://forums.aws.amazon.com/thread.jspa?threadID=324227&tstart=0

[malammar]

For anyone else arriving here, AL2 on Docker no longer needs a custom config in order to stream logs.

By default, all logs go to:

/aws/elasticbeanstalk/{your-env-name}/var/log/eb-docker/containers/eb-current-app/stdouterr.log