airbnb/streamalert

New AWS account sending CloudTrail logs causes classifier error

0xdabbad00 opened this issue · 1 comments

Background

Have you tried pinging us on Slack? Yes

Are you on the latest version of StreamAlert? I think so.

Description

At a company that uses StreamAlert, they had some rogue accounts (like a random developer account) that was made to adhere to company policy. This included having their CloudTrail logs sent to the same S3 bucket as the rest of the company's CloudTrail logs. When this happened, a classifier error occurred.

Looking in the logs, the error mentions S3 object CLOUDTRAIL_BUCKET/AWSLogs/000000000000/CloudTrail/ has an invalid size and cannot be downloadedfrom S3: 0.0KB

The error is in _check_size in stream_alert/classifier/payload/s3.py

Steps to Reproduce

Configure a new AWS account to send it's CloudTrail logs to an S3 bucket that StreamAlert is watching for CloudTrail logs. For testing, you can probably turn off CloudTrail, delete/move the existing logs, then turn on CloudTrail.

Desired Change

I would like to not have error occur under this situation. This situation is going to occur more frequently as companies create more and more AWS accounts or make acquisitions that result in new accounts that need to adhere to company policy.

fixed with #1284