/aws-privatelink-interface-endpoint-monitoring

Primary LanguagePythonMIT No AttributionMIT-0

Monitor network throughput of interface VPC endpoints using Amazon CloudWatch

Ref: https://aws.amazon.com/blogs/mt/monitor-network-throughput-of-interface-vpc-endpoints-using-amazon-cloudwatch/

Prerequisites

To deploy the solution, you need the following:

Deploy the solution using Terraform

The Terraform template has the following input parameters, which you can modify as appropriate for your use case.
Parameter Variable Default Description
AWS Region aws_region sa-east-1 The AWS Region to be used for deployment.
Amazon VPC Id vpc_id The ID of the VPC to be monitored.
Alarm Critical Threshold (Bytes) alarm_critical_threshholdbytes 76504104960 The monitoring threshold, in bytes, for critical alarms.
Alarm Threshhold (Bytes) alarm_threshholdbytes 56371445760 The monitoring threshold, in bytes, for initial alarms.
CloudWatch Log Group cloudwatch_loggroup vpcendpointloggroup The name of the CloudWatch log group name that will capture flow log data.
CloudWatch Metric NameSpace name_space vpcendpoint The CloudWatch metric namespace that will collect metrics for all endpoint interfaces.
SNS Topic ARN for Alarm notification sns_topic_arn The ARN of the SNS topic configured for the CloudWatch alarm.
Log Processing Interval (Min) timerange_min 1 The duration, in minutes, the Lambda function will use to capture log data from the CloudWatch log group.

Deploying the Solution

Using AWS CLI to deploy the template

Run the following command in your AWS CLI environment: 
$ git clone https://github.com/aws-samples/aws-privatelink-interface-endpoint-monitoring
$ terraform init
#Modify variables.tf input parameters as per environment needs by referring above table 
$ terraform plan
$ terraform apply
It will take approximately ~10 minutes to deploy the solution. Upon a successful deployment of the template, the following resources are created:
  • An IAM role, vpcflowlogcwrole, allows VPC Flow Logs to be written to CloudWatch Logs.
  • A VPC flow log where records are stored in this format:
${interface-id} ${bytes} ${subnet-id} ${vpc-id} ${account-id}
interface-id The ID of the network interface for which the traffic is recorded.
bytes The number of bytes transferred during the flow.
subnet-id The ID of the subnet that contains the network interface for which the traffic is recorded.
vpc-id The ID of the VPC that contains the network interface for which the traffic is recorded.
account-id The AWS account ID of the owner of the source network interface for which traffic is recorded.
  • A CloudWatch log group, which captures flow log data in an individual log stream for each interface VPC endpoint.
  • An EventBridge rule that triggers the Lambda function at scheduled intervals.
  • A Lambda function written in Python with environment variables as per user-defined parameters.
  • CloudWatch metrics are created as part of the Lambda function run.
  • Two CloudWatch alarm definitions with each endpoint interface name. Additionally Critical alarms are suffixed with Critical.

Viewing and visualizing metrics

After the solution starts to process event data, you can view the metrics and alarm definitions for the interface endpoints.

Now that your data is available in CloudWatch Metrics, you can create an interactive Amazon CloudWatch dashboard using the instructions mentioned here Creating a CloudWatch dashboard.

The CloudWatch dashboard displays the VPCEndpointMonitor metric.

 Figure 2: CloudWatch dashboard

For every interface endpoint, two alarm definitions are created and configured per user-defined threshold limits.

The Alarms page in the CloudWatch console displays alarms for the VpcEndpointMonitor metric.

Figure 3: Alarms page in the CloudWatch console

Cleanup

To avoid any additional charges after you test the solution, run the following command to delete the resources:

$ terraform destroy

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.