/deploying-siem-with-aws-cdk

Primary LanguagePythonMIT No AttributionMIT-0

Deploying SIEM with AWS CDK

language : Korean

Architecture

img.png

Notices

  • This project includes implementing SIEM on AWS to monitor triggered events in the Centralized log bucket.
  • For Log Source configuration, refer to here to see how to configure Log source for AWS Services.
  • The list of supported Log Type is as follows.
AWS Service Log
Security, Identity, & Compliance AWS Security Hub Security Hub findings
GuardDuty findings
Amazon Macie findings
Amazon Inspector findings
AWS IAM Access Analyzer findings
Security, Identity, & Compliance AWS WAF AWS WAF Web ACL traffic information
AWS WAF Classic Web ACL traffic information
Security, Identity, & Compliance Amazon GuardDuty GuardDuty findings
Security, Identity, & Compliance AWS Network Firewall Flow logs
Alert logs
Management & Governance AWS CloudTrail CloudTrail Log Event
Networking & Content Delivery Amazon CloudFront Standard access log
Real-time log
Networking & Content Delivery Amazon Route 53 Resolver VPC DNS query log
Networking & Content Delivery Amazon Virtual Private Cloud (Amazon VPC) VPC Flow Logs (Version5)
Networking & Content Delivery Elastic Load Balancing Application Load Balancer access logs
Network Load Balancer access logs
Classic Load Balancer access logs
Storage Amazon Simple Storage Service (Amazon S3) access log
Database Amazon Relational Database Service (Amazon RDS)
(Experimental Support)		 Amazon Aurora(MySQL)
Amazon Aurora(PostgreSQL)
Amazon RDS for MariaDB
Amazon RDS for MySQL
Amazon RDS for PostgreSQL
Analytics Amazon Managed Streaming for Apache Kafka (Amazon MSK) Broker log
Compute Linux OS
via CloudWatch Logs		 /var/log/messages
/var/log/secure
Containers Amazon Elastic Container Service (Amazon ECS)
via FireLens		 Framework only

Experimental Support: We may change field type, normalization and something in the future.

1. Setting Up CDK Execution Environment

1.1. On Local

git clone https://github.com/aws-samples/deploying-siem-with-aws-cdk.git
cd deploying-siem-with-aws-cdk/
npm install

1.2. With EC2

  1. Launch an Intance with Amazon Linux 2 AMI on EC2 console. (select higher instance type than t2.micro)
  2. Create EC2 role with Admin Policy and attach it to the EC2.
  3. Connect the instance.
  4. Use the script below to install the required modules and clone repository. 
     sudo yum groups mark install -y "Development Tools"
 sudo yum install -y amazon-linux-extras
 sudo amazon-linux-extras enable python3.8
 sudo yum install -y python38 python38-devel git jq
 sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
 sudo update-alternatives --install /usr/bin/pip3 pip3 /usr/bin/pip3.8 1
 git clone https://github.com/aws-samples/deploying-siem-with-aws-cdk.git

2. Set environment variables

export CDK_DEFAULT_ACCOUNT=<AWS_ACCOUNT> # your AWS account
export AWS_DEFAULT_REGION=<AWS_REGION> # region where the distributable is deployed

Example)

export CDK_DEFAULT_ACCOUNT=888888888888
export AWS_DEFAULT_REGION=ap-northeast-2

3. Create Lambda Deployment Packages

Lambda in this project uses a 3rd party library GeoLite2. Execute the below to download the library and create deployment packages locally.

cd deploying-siem-with-aws-cdk/deployment/
chmod +x ./step1-build-lambda-pkg.sh && ./step1-build-lambda-pkg.sh

4. Set up CDK environment

The script below installs the software required to run aws-cdk. When proceeding locally "Do you really continue?" is exposed, enter 'y' and proceed.

chmod +x ./step2-setup-cdk-env.sh && ./step2-setup-cdk-env.sh
source ~/.bash_profile

5. CDK bootstrap

5-1. Update cdk.json

You can make the necessary changes by referring to the following information. s3_bucket_name 의 log, snapshot, geo are mandatory entries.

Parameter Initial value Description
resource_suffix [blank] suffix to prevent duplication error of resources when re-deploy
aes_domain_name aes-siem Amazon Elasticsearch Service's domain name
s3_bucket_name 3 S3 buckets to be created: You need to update [AWS Account ID] below
*log aes-siem-[AWS Account ID]-log Bucket where centralized logs are stored
*snapshot aes-siem-[AWS Account ID]-snapshot Bucket where ndjson imported is stored
*geo aes-siem-[AWS Account ID]-geo Bucket for which GeoIp mmdb will be downloaded
kms_cmk_alias aes-siem-key AWS KMS CMK(customer-managed key) alias

5-2. validate json

After running, json is displayed and no errors are present, then normal.

cdk context  --j

5-3. cdk bootstrap

cd ..
cdk bootstrap
  • If an error occurs, verify that the EC2 instance's role has Administrator.

6. CDK deploy

6-1. without GeoLite2

cdk deploy

6-2. with GeoLite2 (optional)

If you want to use the geo information through GeoIP as shown the dashboard below in Kibana, follow the procedure below. img_1.png

  1. Enter the license key using the GeoLite2LicenseKey parameter with cdk deploy. 
    cdk deploy --parameters GeoLite2LicenseKey=xxxxxxxxxxxxxxxx
  2. You can create the license key here
    • Sign Up for GeoLite2 -> Login -> Generate a License Key
  3. The deployment takes about 20 minutes.
  4. Test aes-siem-geoip-downloader lambda (with GeoLite2)
    1. Go to Lambda console
    2. Go to Test tab.
    3. Click Test.
    4. Go to S3 condole and see if GeoLite2 folder is created in aes-siem-[AWS Account ID]-geo bucket.

6-3. with AllowedSourceIpAddresses in Elasticsearch Service (optional)

cdk deploy --parameters AllowedSourceIpAddresses="10.0.0.0/8 0.0.0.0" --parameters GeoLite2LicenseKey=xxxxxxxxxxxxxxxx

7. Update ES Access policy

  1. Go to Elasticsearch Service console
  2. Click domain aes-siem > Actions > Modify access policy
  3. Specify the list to allow in aws:SourceIp

8. Go to Kibana

  1. See Outputs tab in the stack you delployed
  2. Go to the KibanaUrl (accessible only from sourceIp specified in #8)
  3. Log in with KibanaAdmin/KibanaPassword in the Outputs tab
    • If you cannot log in with the KibanaAdmin/KibanaPassword, create a master user with aes-siem domain > Actions > Modify authentication in Elasticsearch Service console
  4. Select Global as your tenant
  5. Select Dashboard Menu and verify that the dashboards are configured per AWS services
    • If you proceed with "#6-1 without GeoLite2", it is normal that the data on Geo related panel is not displayed

9. Cleanup

  1. Go to CloudFormation console and click 'delete stack'
  2. The resources below must be deleted by moving to the console of the service.
    • Amazon ES domain: aes-siem{resource_suffix}
    • Amazon S3 bucket: aes-siem-[AWS_Account]-log{resource_suffix}
    • Amazon S3 bucket: aes-siem-[AWS_Account]-snapshot{resource_suffix}
    • Amazon S3 bucket: aes-siem-[AWS_Account]-geo{resource_suffix}
    • AWS KMS customer-managed key: aes-siem-key{resource_suffix}
      • Note: If you delete the customer-managed key (CMK), you will not be able to read the logs that you encrypted using it.

10. Redeploy

  • To redeploy this stack, update the resource_suffix in cdk.json and proceed from #6

License

This product uses GeoLite2 Data created by MaxMind and is licensed under https://www.maxmind.com/en/geolite2/eula incorporating CC-BY-SA, available from https://www.maxmind.com. This product uses ndjson and is licensed under BSD-3-Clause, available from http://ndjson.org/