/CloudWatchLogs2Pandas

Library to query CloudWatch Logs and create Pandas DataFrame from the results

Primary LanguageJupyter NotebookApache License 2.0Apache-2.0

CloudWatchLogs2Pandas

This file will become your README and also the index of your documentation.

Install

pip install CloudWatchLogs2Pandas

How to use

You need to build your query based on the documentations here:

query = 'fields @timestamp, @message | sort @timestamp desc | limit 5'
log_group = '/aws/apigateway/welcome'
cwq = CloudWatchQuery(query, log_group)

When executing the query against the log group, the default time range of of 14 days.

cwq.get_df()
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>

In case you have no results, you can modify the time range. For example for Last Year you can use:

cwq.get_df(days_start=365)
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
timestamp message ptr
0 2021-12-20 11:03:13.568 Cloudwatch logs enabled for API Gateway CmAKKAokMzI3MTYzODM3MDU1Oi9hd3MvYXBpZ2F0ZXdheS...
1 2021-11-24 15:10:19.572 Cloudwatch logs enabled for API Gateway CmAKKAokMzI3MTYzODM3MDU1Oi9hd3MvYXBpZ2F0ZXdheS...
2 2021-11-01 15:57:57.790 Cloudwatch logs enabled for API Gateway CmAKKAokMzI3MTYzODM3MDU1Oi9hd3MvYXBpZ2F0ZXdheS...
3 2021-11-01 15:37:36.574 Cloudwatch logs enabled for API Gateway CmAKKAokMzI3MTYzODM3MDU1Oi9hd3MvYXBpZ2F0ZXdheS...
4 2021-10-30 23:10:28.807 Cloudwatch logs enabled for API Gateway CmAKKAokMzI3MTYzODM3MDU1Oi9hd3MvYXBpZ2F0ZXdheS...

Additional configurations

You can set the AWS region with:

cwq.set_region("us-west-2")

More complicated time range:

cwq.get_df(days_start=2, hours_start=4, hours_end=4)
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>