Usage Analyzer for Amazon CodeWhisperer

The Amazon CodeWhisperer usage analyzer is used to perform analysis on code suggestions and security scans for Organizations using the CodeWhisperer professional tier.

Once deployed all code suggestions and security scans will publish metrics into a data pipeline, which will make the information readily available to query via Amazon Athena. With these metrics you can write queries that are capable of reporting on usage and attributing total spend to a particular user or group.

Deploying

Please note that this solution will need to be deployed into the same account as your CodeWhisperer Professional profile was configured. Currently this stack must be deployed in us-east-1 as CodeWhisperer Professional events are only accessible from this region

Clone the repository
Install the AWS Cloud Developer Kit (AWS CDK). Full instructions can be found here.

npm install -g aws-cdk

Install dependencies

pip install -r requirements.txt

Bootstrap CDK

cdk bootstrap aws://ACCOUNT-NUMBER/REGION

(Optional) Access the AWS IAM Identity Center in the console. Under the Application assignments menu on the left hand side select Applications. Select the CodeWhispererDefaultProfile application. For each group click on the Group name and make a note of the Group ID. Collate them together using the below command.

export SSO_GROUP_IDS="GROUP_ID_1,GROUP_ID2,GROUP_ID_N"

(Optional) If your SSO region is different than the region this stack is being deployed in, set the following environment variables

export SSO_REGION="REGION_CODE"

Deploy the CDK Stack in your AWS account (ignoring the context flag if you did not complete step 5 or step 6)

cdk deploy --context sso_group_ids=$SSO_GROUP_IDS --context sso_region=$SSO_REGION

Example Queries

Below are some example queries which can be used to get common results. If you are not seeing all of the expected results, please ensure that the AWS Glue Data Crawler has triggered for the day. It is scheduled to run once per hour to identify new partitions.

Please note that the following fields require specifying at least one group in the SSO_GROUP_IDS variable during deployment: "user_name", "group_id", "group_name"

How many unique users used CodeWhisperer this month?

SELECT
    COUNT(DISTINCT "user_id") as total_unique_users
FROM
    "codewhisperer_events"."codewhispererevents"
WHERE
    year='2023'
AND
    month='08'
LIMIT 10;

Who has used CodeWhisperer this month?

SELECT
    DISTINCT "user_name"
FROM
    "codewhisperer_events"."codewhispererevents"
WHERE
    year='2023'
AND
    month='08'
LIMIT 10;

What has been my most popular language?

SELECT
    "programming_language",
    COUNT(*) as "language_count"
FROM
    "codewhisperer_events"."codewhispererevents"
WHERE
    year='2023'
AND
    month='08'
AND
    "programming_language" IS NOT NULL
GROUP BY
    "programming_language"
ORDER BY
    "language_count" DESC
LIMIT 10;

How are unique users split across groups?

SELECT
    "group_name",
    COUNT(DISTINCT "user_id") as "total_users"
FROM
    "codewhisperer_events"."codewhispererevents"
WHERE
    year='2023'
AND
    month='08'
GROUP BY
    "group_name",
    "user_id"
LIMIT 10;

How many security scans have we run?

SELECT
    "event_type",
    COUNT(*) as "total_scans"
FROM
    "codewhisperer_events"."codewhispererevents"
WHERE
    year='2023'
AND
    month='08'
AND
    "event_type"='SecurityScanInvocation'
GROUP BY
    "event_type"
LIMIT 10;

Cost Estimation

There are a number of services involved in the architecture (please see above). The following pricing items will need to be considered.

AWS CloudTrail - CodeWhisperer data events are used to capture the elements for this solution. Trails with data events carry charge to be delivered to an S3 bucket.
Amazon S3 - S3 charges are used for storing the CloudTrail events and the processed events. If you do not need the data to persist forever, consider your storage lifecycle.
Amazon Kinesis Data Firehose - Firehose charges per ingestion per GB. This will vary by the total events (code generation and security scans) generated in CodeWhisperer
AWS Lambda - The Lambda function included is used to process requests in Firehose. Each function invoked will bundle multiple records at one time. The Lmabda is configured to use Arm64 processor with a memory configuration of 512MB.
AWS Glue - The Glue Crawler used in this solution runs once every 6 hours
Amazon Athena - Athena pricing will vary based on query usage over the month

To get a more accurate understanding, please use the AWS Pricing Calculator providing your estimated usage.

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.

aws-samples/amazon-codewhisperer-usage-analyzer