We will setup ML Detect feature using AWS CLI using following guide and will also setup scripts on how to create anamoly from device side for us to test the system.
Lets first setup our ML Detect Security profile and go through steps on attaching to specific IoT things (devices) / group.
Before we setup using AWS CLI, please make sure you have AWS CLI setup for your environment (setting up CLI for your environment further instructions) and configure correctly (further instructions). The minimum required version of the CLI for setting this up is is at least 1.9.12 or higher ****(including aws cli v2)
You can test AWS CLI to make sure its all setup correctly
aws help
Step 1 [name] Once you have setup / updated your CLI to minimum required version, we will start by creating Security Profile using following snippet as below:
Create Security Profile command:
aws ml-iot create-security-profile \
--security-profile-name <value> \
[--security-profile-description <value>] \
[--behaviors <value>] \
[--alert-targets <value>] \
[--additional-metrics-to-retain <value>] \
[--tags <value>] \
[--cli-input-json <value>] \
[--generate-cli-skeleton]
Sample using eu-west-1 as region and Profile name ML_Detect_profile:
aws ml-iot create-security-profile \
--security-profile-name ML_Detect_profile \
--behaviors \
'[{
"name": "num-messages-sent-ml-behavior",
"metric": "aws:num-messages-sent",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "MEDIUM"
}
},
"suppressAlerts": true
},
{
"name": "num-authorization-failures-ml-behavior",
"metric": "aws:num-authorization-failures",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "MEDIUM"
}
},
"suppressAlerts": true
},
{
"name": "num-connection-attempts-ml-behavior",
"metric": "aws:num-connection-attempts",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "MEDIUM"
}
},
"suppressAlerts": true
},
{
"name": "num-disconnects-ml-behavior",
"metric": "aws:num-disconnects",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "MEDIUM"
}
},
"suppressAlerts": true
},
{
"name": "num-msg-received-ml-behavior",
"metric": "aws:num-messages-received",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "MEDIUM"
}
},
"suppressAlerts": true
},
{
"name": "msg-byte-size-ml-behavior",
"metric": "aws:message-byte-size",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "MEDIUM"
}
},
"suppressAlerts": true
}]' \
--region eu-west-1 --endpoint-url https://iot.eu-west-1.amazonaws.com
We will use the get-behavior-model-training-summaries command to obtain the status of our model. Please update the region which you are using. Region details can be found here and CLI to obtain detail on region here
ML Model training summary command:
aws ml-iot get-behavior-model-training-summaries
[--security-profile-name <value>]
[--max-results <value>]
[--next-token <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--cli-auto-prompt <value>]
Sample for Ireland Region i will run this as follows with profile name being ML_Detect_profile:
aws ml-iot get-behavior-model-training-summaries
-security-profile-name ML_Detect_profile
--region eu-west-1
--endpoint-url https://iot.eu-west-1.amazonaws.com/
we should see following output as below:
{
"summaries": [
{
"securityProfileName": "ML_Detect_profile",
"behaviorName": "num-messages-sent-ml-behavior",
"behaviorCreateDate": 1600763457.948,
"modelStatus": "PENDING_BUILD",
"datapointsCollectionPercentage": 0.0
},
{
"securityProfileName": "ML_Detect_profile",
"behaviorName": "num-authorization-failures-ml-behavior",
"behaviorCreateDate": 1600763457.948,
"modelStatus": "PENDING_BUILD",
"datapointsCollectionPercentage": 0.0
},
{
"securityProfileName": "ML_Detect_profile",
"behaviorName": "num-connection-attempts-ml-behavior",
"behaviorCreateDate": 1600763457.948,
"modelStatus": "PENDING_BUILD",
"datapointsCollectionPercentage": 0.0
},
{
"securityProfileName": "ML_Detect_profile",
"behaviorName": "num-disconnects-ml-behavior",
"behaviorCreateDate": 1600763457.948,
"modelStatus": "PENDING_BUILD",
"datapointsCollectionPercentage": 0.0
}
]
}
Same as console we can see attribute modelStatus: being PENDING_BUILD, once it becomes active we will see the ACTIVE state.
Let’s view Active violations using list-active-violations ****with further options i.e. max result being 2 Please update the region which you are using. Region details can be found here and CLI to obtain detail on region here
Command for Active violations:
aws ml-iot list-active-violations
[--thing-name <value>]
[--security-profile-name <value>]
[--behavior-criteria-type <value>]
[--list-suppressed-alerts | --no-list-suppressed-alerts]
[--next-token <value>]
[--max-results <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--cli-auto-prompt <value>]
Command for Violation Events:
aws ml-iot list-violation-events
--start-time <value>
--end-time <value>
[--thing-name <value>]
[--security-profile-name <value>]
[--behavior-criteria-type <value>]
[--list-suppressed-alerts | --no-list-suppressed-alerts]
[--next-token <value>]
[--max-results <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--cli-auto-prompt <value>]
Sample command for active violations for Ireland region:
aws ml-iot list-active-violations
--max-results 2
-—region eu-west-1
Sample command for active historical violation events with specific start time and end time i.e.
aws ml-iot list-violation-events \
--start-time 1599500533 --end-time 1600796533 \
--max-results 2 \
--region eu-west-1
--endpoint-url https://iot.eu-west-1.amazonaws.com/
Sample output:
{
"violationEvents": [
{
"violationId": "1448be98c09c3d4ab7cb9b6f3ece65d6",
"thingName": "ddml12",
"securityProfileName": "ML_Detect_profile",
"behavior": {
"name": "LowConfidence_MladBehavior_MessagesReceived",
"metric": "aws:num-messages-received",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel": "LOW"
}
},
"suppressAlerts": false
},
"violationEventType": "alarm-cleared",
"violationEventTime": 1600780245.29
}
}
Similarly using CLI we can update Confidence level and not to suppress alerts using sample command as below (using EU Ireland as region): Please update the region which you are using. Region details can be found here and CLI to obtain detail on region hereand update Security Profile name
Command for updating Security profile:
aws ml-iot update-security-profile
--security-profile-name <value>
[—security-profile-description <value>]
[—behaviors <value>]
[—alert-targets <value>]
[—additional-metrics-to-retain <value>]
[—additional-metrics-to-retain-v2 <value>]
[--delete-behaviors | —no-delete-behaviors]
[--delete-alert-targets | —no-delete-alert-targets]
[--delete-additional-metrics-to-retain | —no-delete-additional-metrics-to-retain]
[—expected-version <value>]
[--cli-input-json | —cli-input-yaml]
[—generate-cli-skeleton <value>]
[—cli-auto-prompt <value>]
Sample Command for updating the security profile:
aws ml-iot update-security-profile \
--security-profile-name ML_Detect_profile \
--behaviors \
'[{
"name": "num-messages-sent-ml-behavior",
"metric": "aws:num-messages-sent",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "HIGH"
}
},
"suppressAlerts": false
},
{
"name": "num-authorization-failures-ml-behavior",
"metric": "aws:num-authorization-failures",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "HIGH"
}
},
"suppressAlerts": false
},
{
"name": "num-connection-attempts-ml-behavior",
"metric": "aws:num-connection-attempts",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "MEDIUM"
}
},
"suppressAlerts": false
},
{
"name": "num-disconnects-ml-behavior",
"metric": "aws:num-disconnects",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "LOW"
}
},
"suppressAlerts": true
},
{
"name": "num-msg-received-ml-behavior",
"metric": "aws:num-messages-received",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "MEDIUM"
}
},
"suppressAlerts": true
},
{
"name": "msg-byte-size-ml-behavior",
"metric": "aws:message-byte-size",
"criteria": {
"mlDetectionConfig": {
"confidenceLevel" : "HIGH"
}
},
"suppressAlerts": false
}]' \
--region eu-west-1 --endpoint-url https://iot.eu-west-1.amazonaws.com
Lets look at creating mitigation action using CLI
Command for creating mitigation action:
aws ml-iot create-mitigation-action
--action-name <value>
--role-arn <value>
--action-params <value>
[--tags <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--cli-auto-prompt <value>]