Azure/Azure-Network-Security

WAF Triage Workbook error

BigRollTide opened this issue · 4 comments

BigRollTide commented

Describe the bug
Error received when connecting to Log Analytics Workspace. There is no policyScopeName_s table.

Reproduce
Steps to reproduce the behavior:
I deploy the Application Gateway WAF triage workbook, and chose my Subscription and Log Analytics Workspace
I then get an error in the middle (Select the scope of the WAF policy) and received the error "project-reorder: Failed to resolve attribute as a column entity: policyScopeName_s..."

Expected behavior
Expected the workbook to show some data

Screenshots
AGW-WAFTriage-Error

Environment- if applicable
Used the link to Deploy to Azure

Desktop (please complete the following information if applicable):

  • OS: Windows 10
  • Browser: Chrome and Edge get same results

Additional context
There is no policyScopeName_s table in Log Analytics. We are using only v2 APGs and are checking all boxes under Diagnostics Settings to export
This is a Log Analytics Workspace for most of our Azure resources - so it has diagnostic logs for both a few FD WAFs and a few AG WAFs. I also have setup the Azure Monitor Workbook for WAF with no issues pointing to the same Log Analytics Workspace.

tobystic commented

Thank you for the feedback. We are aware of this update request and working on making the reference links aligned

tobystic commented

@xstof is there logic to capture both CRS and DRS for the URL reference for the signatures or do we need a DRS version of the workbook entirely?

BigRollTide commented

I have updated the code for the new logging. Here is a new template

{ "contentVersion": "1.0.0.0", "parameters": { "workbookDisplayName": { "type": "string", "defaultValue": "Application Gateway WAF Triage", "metadata": { "description": "The friendly name for the workbook that is used in the Gallery or Saved List. This name must be unique within a resource group." } }, "workbookSourceId": { "type": "string", "defaultValue": "", "metadata": { "description": "The id of resource instance to which the workbook will be associated" } }, "workbookId": { "type": "string", "defaultValue": "[newGuid()]", "metadata": { "description": "The unique guid for this workbook instance" } } }, "variables": { "workbookContent": { "version": "Notebook/1.0", "items": [ { "type": 1, "content": { "json": "## Application Gateway WAF triage workbook\n---\n\nThis workbook will help you triage Application Gateway WAF violations." }, "name": "text - 2" }, { "type": 9, "content": { "version": "KqlParameterItem/1.0", "parameters": [ { "id": "63ea93e5-3270-4447-9572-ad9e62095e7b", "version": "KqlParameterItem/1.0", "name": "subscription", "label": "Subscription", "type": 6, "isRequired": true, "multiSelect": true, "quote": "'", "delimiter": ",", "value": [ "/subscriptions/651dc44c-5d8e-48da-8cd3-cd79224ac290" ], "typeSettings": { "additionalResourceOptions": [], "includeAll": false } }, { "id": "12476248-81b7-46af-a9b4-790f93306442", "version": "KqlParameterItem/1.0", "name": "workspace", "label": "Workspace", "type": 5, "isRequired": true, "query": "where type =~ 'microsoft.operationalinsights/workspaces'\r\n| summarize by id, name\r\n| project id", "crossComponentResources": [ "{subscription}" ], "value": "/subscriptions/651dc44c-5d8e-48da-8cd3-cd79224ac290/resourceGroups/xstof-appgw/providers/Microsoft.OperationalInsights/workspaces/xstof-appgw", "typeSettings": { "additionalResourceOptions": [] }, "queryType": 1, "resourceType": "microsoft.resourcegraph/resources" }, { "id": "f451ddc5-00e3-4fe7-908e-22d0be9ab59c", "version": "KqlParameterItem/1.0", "name": "detectionTime", "label": "Detection Time", "type": 4, "isRequired": true, "value": { "durationMs": 14400000 }, "typeSettings": { "selectableValues": [ { "durationMs": 300000 }, { "durationMs": 900000 }, { "durationMs": 1800000 }, { "durationMs": 3600000 }, { "durationMs": 14400000 }, { "durationMs": 43200000 }, { "durationMs": 86400000 }, { "durationMs": 172800000 }, { "durationMs": 259200000 }, { "durationMs": 604800000 }, { "durationMs": 1209600000 }, { "durationMs": 2419200000 }, { "durationMs": 2592000000 }, { "durationMs": 5184000000 }, { "durationMs": 7776000000 } ] }, "timeContext": { "durationMs": 86400000 } } ], "style": "pills", "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces" }, "name": "parameters - 2" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics \n| where Category == \"ApplicationGatewayFirewallLog\"\n| where TimeGenerated {detectionTime}\n| extend Id = new_guid()\n| extend policyScopeName = AdditionalFields.policyScopeName\n//| project-reorder Id, ResourceId, action_s\n| project-reorder Id, ResourceId, policyScopeName, action_s\n//| summarize count() by ResourceId, policyScopeName, action_s\n//| evaluate pivot(action_s, count(), ResourceId)\n| evaluate pivot(action_s, count(), ResourceId, policyScopeName)", "size": 1, "title": "Select the scope of the WAF policy", "exportedParameters": [ { "fieldName": "policyScopeName", "parameterName": "PolicyScope", "parameterType": 1 }, { "fieldName": "ResourceId", "parameterName": "ResourceId", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "visualization": "table", "gridSettings": { "formatters": [ { "columnMatch": "$gen_group", "formatter": 13, "formatOptions": { "linkTarget": null, "showIcon": true } }, { "columnMatch": "ResourceId", "formatter": 5 }, { "columnMatch": "Matched", "formatter": 8, "formatOptions": { "palette": "greenRed" } }, { "columnMatch": "Blocked", "formatter": 8, "formatOptions": { "palette": "greenRed" } } ], "rowLimit": 500, "hierarchySettings": { "treeType": 1, "groupBy": [ "ResourceId" ] }, "labelSettings": [ { "columnId": "ResourceId", "label": "App Gateway" }, { "columnId": "policyScopeName", "label": "Policy Scope" } ] }, "tileSettings": { "showBorder": false } }, "name": "query-app-gateway-resources" }, { "type": 11, "content": { "version": "LinkItem/1.0", "style": "tabs", "links": [ { "id": "14e61d36-d59a-405d-8363-ac1aa8a2f491", "cellValue": "SelectedTabPage", "linkTarget": "parameter", "linkLabel": "Triage by Rule", "subTarget": "triage-by-rule", "preText": "Triage by Rule", "style": "link" }, { "id": "93464fc4-d183-4d74-9e06-111c6de081a9", "cellValue": "SelectedTabPage", "linkTarget": "parameter", "linkLabel": "Triage by URL", "subTarget": "triage-by-url", "preText": "Triage ", "style": "link" } ] }, "name": "TabNavigation" }, { "type": 12, "content": { "version": "NotebookGroup/1.0", "groupType": "editable", "items": [ { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics \r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where TimeGenerated {detectionTime}\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| extend ruleSetType_s = AdditionalFields.ruleSetType\r\n| extend ruleSetVersion_s = AdditionalFields.ruleSetVersion\r\n| extend ruleId_s = AdditionalFields.ruleId\r\n| extend hostname_s = AdditionalFields.hostname\r\n| extend details_message_s = AdditionalFields.details_message\r\n| extend details_file_s = AdditionalFields.details_file\r\n| extend details_line_s = AdditionalFields.details_line\r\n| summarize Count = count() by tostring(ruleSetType_s), tostring(ruleSetVersion_s), tostring(ruleId_s), tostring(details_file_s), tostring(details_line_s), action_s\r\n| order by Count desc\r\n| project-reorder Count\r\n| extend normalizedRuleSetVersion = case(isempty(ruleSetVersion_s), \"\",\r\n // old engine: ruleSetVersion_s looks like \"3.1.0\" while type is \"OWASP_CRS\" => leave untouched, can use this as-is\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.1.0/rules/REQUEST-913-SCANNER-DETECTION.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+.[0-9]+\", ruleSetVersion_s,\r\n // new engine: ruleSetVersion_s looks like \"3.2\" while type is \"OWASP CRS\" => need a version that looks like \"3.2.0\"\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.2.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+\", strcat(ruleSetVersion_s, \".0\"),\r\n \"\")\r\n| extend normalizedDetailsFile = case( ruleSetType_s == \"OWASP_CRS\", replace(@'rules/', '', details_file_s), // old engine: details_file_s looks like \"rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP_CRS\"\r\n ruleSetType_s == \"OWASP CRS\", details_file_s, // new engine: details_file_s looks like \"REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP CRS\"\r\n details_file_s)\r\n| extend Url = iif(isnotempty(ruleSetVersion_s) and ruleSetVersion_s !startswith \"1\", strcat('https://github.com/coreruleset/coreruleset/blob/', 'v', normalizedRuleSetVersion, '/rules/', normalizedDetailsFile, '#L', details_line_s), '')\r\n// add sparklines\r\n| join kind=inner (AzureDiagnostics | where Category == \"ApplicationGatewayFirewallLog\" | where TimeGenerated {detectionTime} | where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| extend ruleId_s = AdditionalFields.ruleId\r\n| make-series Trend = count() on TimeGenerated from {detectionTime:start} to {detectionTime:end} step (time({detectionTime:seconds} seconds) / 20) by tostring(ruleId_s)\r\n ) on ruleId_s\r\n| extend Id = new_guid()\r\n| project Count,ruleSetVersion_s,ruleId_s,action_s,Trend,Url,Id", "size": 0, "showAnalytics": true, "title": "Rules that got triggered", "exportedParameters": [ { "fieldName": "ruleId_s", "parameterName": "RuleId" }, { "fieldName": "Url", "parameterName": "SpiderLabsUrl", "parameterType": 1 }, { "fieldName": "ruleSetVersion_s", "parameterName": "RuleSetVersion", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "gridSettings": { "formatters": [ { "columnMatch": "Count", "formatter": 8, "formatOptions": { "palette": "greenRed" } }, { "columnMatch": "ruleId_s", "formatter": 5 }, { "columnMatch": "Trend", "formatter": 10, "formatOptions": { "palette": "yellowOrangeRed" } }, { "columnMatch": "Url", "formatter": 7, "formatOptions": { "linkTarget": "Url", "customColumnWidthSetting": "100%" }, "numberFormat": { "unit": 0, "options": { "style": "decimal" }, "emptyValCustomText": "No link for Custom or Bot rules" } }, { "columnMatch": "Id", "formatter": 5 } ], "sortBy": [ { "itemKey": "$gen_heatmap_Count_0", "sortOrder": 2 } ], "labelSettings": [ { "columnId": "ruleSetVersion_s", "label": "Ruleset Version" }, { "columnId": "ruleId_s", "label": "Rule Id" }, { "columnId": "action_s", "label": "Action" } ] }, "sortBy": [ { "itemKey": "$gen_heatmap_Count_0", "sortOrder": 2 } ] }, "customWidth": "100", "name": "query - violated rules" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics\r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}' and AdditionalFields.ruleId == '{RuleId}' and AdditionalFields.ruleSetVersion == '{RuleSetVersion}'\r\n| where TimeGenerated {detectionTime}\r\n// at this point AdditionalFields.hostname can still contain both hostname and port; for example: www.myhost.com:80 -> we need to split this up to match on just the host later\r\n| extend hostname_s = AdditionalFields.hostname\r\n| extend splittedhost = split(hostname_s, ':')\r\n| extend hostname_s_noport = tostring(splittedhost[0]) //override AdditionalFields.hostname to just contain host; drop the port section\r\n| extend port = tostring(splittedhost[1])\r\n| distinct tostring(hostname_s), hostname_s_noport", "size": 0, "title": "Hosts Affected", "exportedParameters": [ { "fieldName": "hostname_s", "parameterName": "HostName" }, { "fieldName": "hostname_s_noport", "parameterName": "HostNameNoPort", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "visualization": "table", "gridSettings": { "formatters": [ { "columnMatch": "hostname_s_noport", "formatter": 5 } ], "labelSettings": [ { "columnId": "hostname_s", "label": "Hostname" } ] } }, "customWidth": "15", "showPin": true, "name": "Affected Hosts" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics\r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where TimeGenerated {detectionTime}\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}' and AdditionalFields.ruleId == '{RuleId}'\r\n| where AdditionalFields.hostname == '{HostName}'\r\n| extend hostname_s = AdditionalFields.hostname\r\n| project-reorder hostname_s, transactionId_g\r\n| join kind=inner (AzureDiagnostics | where Category == \"ApplicationGatewayAccessLog\" | where TimeGenerated {detectionTime}) on transactionId_g\r\n| distinct host_s1, httpMethod_s1, requestUri_s1, originalRequestUriWithArgs_s1\r\n| order by host_s1, requestUri_s1, httpMethod_s1, originalRequestUriWithArgs_s1\r\n| extend id = new_guid()", "size": 0, "showAnalytics": true, "title": "Hosts and urls impacted by selected rule", "exportedParameters": [ { "fieldName": "httpMethod_s1", "parameterName": "Method", "parameterType": 1 }, { "fieldName": "originalRequestUriWithArgs_s1", "parameterName": "FullUri", "parameterType": 1 }, { "fieldName": "requestUri_s1", "parameterName": "RequestUri", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "visualization": "table", "gridSettings": { "formatters": [ { "columnMatch": "requestUri_s", "formatter": 5 }, { "columnMatch": "encodedRequestUri", "formatter": 5 }, { "columnMatch": "id", "formatter": 5 } ], "rowLimit": 1000, "hierarchySettings": { "treeType": 1, "groupBy": [ "requestUri_s1" ], "expandTopLevel": true, "finalBy": "requestUri_s" }, "labelSettings": [ { "columnId": "host_s1", "label": "Hostname" }, { "columnId": "httpMethod_s1", "label": "HTTP Method" }, { "columnId": "requestUri_s1", "label": "Request Path" }, { "columnId": "originalRequestUriWithArgs_s1", "label": "Original Request Uri with Arguments" } ] } }, "customWidth": "60", "name": "affected urls" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics\r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| where TimeGenerated {detectionTime}\r\n| distinct transactionId_g\r\n| join kind=inner (AzureDiagnostics | where Category == \"ApplicationGatewayAccessLog\" | where TimeGenerated {detectionTime}) \r\n on transactionId_g\r\n| where originalHost_s == '{HostNameNoPort}' and httpMethod_s == '{Method}' and requestUri_s == ```{RequestUri}``` and originalRequestUriWithArgs_s == ```{FullUri}```\r\n| project transactionId_g\r\n| extend access_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayAccessLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")\r\n| extend firewall_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayFirewallLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")\r\n//let transIdsWithFirewallLog = AzureDiagnostics\r\n//| where Category == \"ApplicationGatewayFirewallLog\"\r\n//| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n//| where TimeGenerated {detectionTime}\r\n//| distinct transactionId_g;\r\n//AzureDiagnostics\r\n//| where Category == \"ApplicationGatewayAccessLog\"\r\n//| where TimeGenerated {detectionTime}\r\n//| where host_s == '{HostName}' and httpMethod_s == '{Method}' and requestUri_s == '{RequestUri}' and originalRequestUriWithArgs_s == '{FullUri}'\r\n//| distinct transactionId_g\r\n//| where transactionId_g in (transIdsWithFirewallLog) // TODO: how to get tenant id in here?\r\n//| extend access_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayAccessLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")\r\n//| extend firewall_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayFirewallLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")", "size": 0, "title": "Requests on selected host and url", "exportFieldName": "transactionId_g", "exportParameterName": "TransactionId", "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "gridSettings": { "formatters": [ { "columnMatch": "transactionId_g", "formatter": 0, "formatOptions": { "customColumnWidthSetting": "40%" } }, { "columnMatch": "access_log", "formatter": 7, "formatOptions": { "linkTarget": "OpenBlade", "linkLabel": "Find in Access Log", "linkIsContextBlade": true, "bladeOpenContext": { "bladeName": "LogsBlade", "extensionName": "Microsoft_Azure_Monitoring_Logs", "bladeParameters": [ { "name": "resourceId", "source": "parameter", "value": "workspace" }, { "name": "query", "source": "cell", "value": "" } ] } } }, { "columnMatch": "firewall_log", "formatter": 7, "formatOptions": { "linkTarget": "OpenBlade", "linkLabel": "Find in Firewall Log", "linkIsContextBlade": true, "bladeOpenContext": { "bladeName": "LogsBlade", "extensionName": "Microsoft_Azure_Monitoring_Logs", "bladeParameters": [ { "name": "resourceId", "source": "parameter", "value": "workspace" }, { "name": "query", "source": "cell", "value": "" } ] } } } ], "labelSettings": [ { "columnId": "transactionId_g", "label": "TransactionId" }, { "columnId": "access_log", "label": "Access Log" }, { "columnId": "firewall_log", "label": "Firewall Log" } ] } }, "customWidth": "25", "name": "ImpactedRequestsFromSelectedUri" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics \r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where transactionId_g == '{TransactionId}'\r\n| extend Id = new_guid()\r\n| extend ruleSetType_s = AdditionalFields.ruleSetType\r\n| extend ruleSetVersion_s = AdditionalFields.ruleSetVersion\r\n| extend ruleId_s = AdditionalFields.ruleId\r\n| extend hostname_s = AdditionalFields.hostname\r\n| extend details_message_s = AdditionalFields.details_message\r\n| extend details_file_s = AdditionalFields.details_file\r\n| extend details_line_s = AdditionalFields.details_line| project-reorder Id, action_s, ruleSetType_s, ruleSetVersion_s, ruleId_s, Message, hostname_s, requestUri_s, details_message_s, details_data_s, details_file_s, details_line_s, transactionId_g\r\n| extend normalizedRuleSetVersion = case(isempty(ruleSetVersion_s), \"\",\r\n // old engine: ruleSetVersion_s looks like \"3.1.0\" while type is \"OWASP_CRS\" => leave untouched, can use this as-is\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.1.0/rules/REQUEST-913-SCANNER-DETECTION.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+.[0-9]+\", ruleSetVersion_s,\r\n // new engine: ruleSetVersion_s looks like \"3.2\" while type is \"OWASP CRS\" => need a version that looks like \"3.2.0\"\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.2.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+\", strcat(ruleSetVersion_s, \".0\"),\r\n \"\")\r\n| extend normalizedDetailsFile = case( ruleSetType_s == \"OWASP_CRS\", replace(@'rules/', '', tostring(details_file_s)), // old engine: details_file_s looks like \"rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP_CRS\"\r\n ruleSetType_s == \"OWASP CRS\", details_file_s, // new engine: details_file_s looks like \"REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP CRS\"\r\n details_file_s)\r\n| extend Url = iif(isnotempty(ruleSetVersion_s) and ruleSetVersion_s !startswith \"1\", strcat('https://github.com/coreruleset/coreruleset/blob/', 'v', normalizedRuleSetVersion, '/rules/', normalizedDetailsFile, '#L', details_line_s), '')\r\n| project action_s, ruleId_s, Message, details_message_s, details_data_s, TimeGenerated, Url", "size": 0, "title": "Rules that got triggered for selected request", "exportedParameters": [ { "fieldName": "transactionId_g", "parameterName": "TransactionId" }, { "fieldName": "Message", "parameterName": "message", "parameterType": 1 }, { "fieldName": "details_message_s", "parameterName": "details", "parameterType": 1 }, { "fieldName": "details_data_s", "parameterName": "matcheddata", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "visualization": "table", "gridSettings": { "formatters": [ { "columnMatch": "action_s", "formatter": 18, "formatOptions": { "thresholdsOptions": "icons", "thresholdsGrid": [ { "operator": "==", "thresholdValue": "Matched", "representation": "2", "text": "{0}{1}" }, { "operator": "==", "thresholdValue": "Blocked", "representation": "4", "text": "{0}{1}" }, { "operator": "Default", "thresholdValue": null, "representation": "more", "text": "{0}{1}" } ] } }, { "columnMatch": "ruleId_s", "formatter": 18, "formatOptions": { "thresholdsOptions": "colors", "thresholdsGrid": [ { "operator": "==", "thresholdValue": "{RuleId}", "representation": "yellow", "text": "{0}{1}" }, { "operator": "Default", "thresholdValue": null, "text": "{0}{1}" } ] } }, { "columnMatch": "Url", "formatter": 7, "formatOptions": { "linkTarget": "Url", "customColumnWidthSetting": "100%" }, "numberFormat": { "unit": 0, "options": { "style": "decimal" }, "emptyValCustomText": "No link for Custom or Bot rules" } }, { "columnMatch": "statusFromAction", "formatter": 5, "numberFormat": { "unit": 0, "options": { "style": "decimal" } } } ], "labelSettings": [ { "columnId": "action_s", "label": "Action" }, { "columnId": "ruleId_s", "label": "Rule Id" }, { "columnId": "Message", "label": "Message" }, { "columnId": "details_message_s", "label": "Details" }, { "columnId": "details_data_s", "label": "Matched Data" }, { "columnId": "TimeGenerated", "label": "Time Generated" } ] } }, "name": "TriggeredRules" }, { "type": 1, "content": { "json": "## Selection Summary:\r\n\r\nYou were looking for example requests which were impacted by WAF rule with id: **{RuleId}**. \r\n\r\nMore information on the selected rule which impacted this url, can be found on the [Core Rule Set Project](https://coreruleset.org/) GitHub page, here:, here: {SpiderLabsUrl}\r\n\r\nOn the host with name \"*{HostName}*\" and url \"*{FullUri}*\" you have found an HTTP request, impacted by this rule, with transaction id equal to *{TransactionId}*. \r\n\r\nMore info on this request and the selected rule:\r\n- Message: {message}\r\n- Details: {details}\r\n- Matched Data: {matcheddata}" }, "name": "text - 7" } ], "exportParameters": true }, "conditionalVisibility": { "parameterName": "SelectedTabPage", "comparison": "isEqualTo", "value": "triage-by-rule" }, "name": "Triage by rule" }, { "type": 12, "content": { "version": "NotebookGroup/1.0", "groupType": "editable", "items": [ { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics \r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| where TimeGenerated {detectionTime}\r\n| distinct tostring(AdditionalFields.hostname)\r\n| project hostname_s = AdditionalFields_hostname", "size": 0, "title": "Hostnames with entries in Firewall Log", "exportFieldName": "hostname_s", "exportParameterName": "HostName", "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "gridSettings": { "formatters": [ { "columnMatch": "hostname_s", "formatter": 0, "formatOptions": { "customColumnWidthSetting": "100%" } } ], "sortBy": [ { "itemKey": "hostname_s", "sortOrder": 1 } ], "labelSettings": [ { "columnId": "hostname_s", "label": "Hostname" } ] }, "sortBy": [ { "itemKey": "hostname_s", "sortOrder": 1 } ] }, "customWidth": "10", "name": "Hostnames with entries in Firewall Log" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics\r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| where AdditionalFields.hostname == '{HostName}'\r\n| where TimeGenerated {detectionTime}\r\n| join kind=inner (AzureDiagnostics | where Category == \"ApplicationGatewayAccessLog\" | where ResourceId == '{ResourceId}')\r\n on transactionId_g\r\n| distinct requestUri_s, httpMethod_s1, originalRequestUriWithArgs_s1, httpStatus_d1\r\n| extend Id = new_guid()", "size": 0, "showAnalytics": true, "title": "Paths which have triggered firewall rules", "exportedParameters": [ { "fieldName": "requestUri_s", "parameterName": "RequestUri" }, { "fieldName": "originalRequestUriWithArgs_s1", "parameterName": "RequestUriWithArgs", "parameterType": 1 }, { "fieldName": "httpMethod_s1", "parameterName": "HttpMethod", "parameterType": 1 }, { "fieldName": "httpStatus_d1", "parameterName": "HttpStatus", "parameterType": 1 }, { "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "gridSettings": { "formatters": [ { "columnMatch": "encodedRequestUriWithArgs", "formatter": 5 }, { "columnMatch": "Id", "formatter": 5 } ], "filter": true, "hierarchySettings": { "treeType": 1, "groupBy": [ "requestUri_s", "httpMethod_s1" ], "finalBy": "httpStatus_d1" }, "sortBy": [ { "itemKey": "httpStatus_d1", "sortOrder": 1 } ], "labelSettings": [ { "columnId": "httpMethod_s1", "label": "HTTP Method" }, { "columnId": "originalRequestUriWithArgs_s1", "label": "Original Request Uri with Arguments" }, { "columnId": "httpStatus_d1", "label": "HTTP Status" } ] }, "sortBy": [ { "itemKey": "httpStatus_d1", "sortOrder": 1 } ] }, "customWidth": "60", "name": "Paths which have triggered firewall rules" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics\r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n| where ResourceId == '{ResourceId}' and AdditionalFields.policyScopeName == '{PolicyScope}'\r\n| where AdditionalFields.hostname == '{HostName}'\r\n| where TimeGenerated {detectionTime}\r\n| join kind=inner (AzureDiagnostics | where Category == \"ApplicationGatewayAccessLog\" | where ResourceId == '{ResourceId}')\r\n on transactionId_g\r\n| where originalRequestUriWithArgs_s1 == ```{RequestUriWithArgs}``` and httpMethod_s1 == '{HttpMethod}' and httpStatus_d1 == '{HttpStatus}'\r\n| distinct transactionId_g\r\n| extend access_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayAccessLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")\r\n| extend firewall_log = strcat(\"AzureDiagnostics | where Category == \\\"ApplicationGatewayFirewallLog\\\" | where TimeGenerated {detectionTime} | where transactionId_g == \\\"\", transactionId_g, \"\\\"\")", "size": 0, "title": "Requests on selected host and url", "exportFieldName": "transactionId_g", "exportParameterName": "TransactionId", "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "gridSettings": { "formatters": [ { "columnMatch": "transactionId_g", "formatter": 0, "formatOptions": { "customColumnWidthSetting": "40%" } }, { "columnMatch": "access_log", "formatter": 7, "formatOptions": { "linkTarget": "OpenBlade", "linkLabel": "Find in Access Log", "linkIsContextBlade": true, "bladeOpenContext": { "bladeName": "LogsBlade", "extensionName": "Microsoft_Azure_Monitoring_Logs", "bladeParameters": [ { "name": "resourceId", "source": "parameter", "value": "workspace" }, { "name": "query", "source": "cell", "value": "" } ] } } }, { "columnMatch": "firewall_log", "formatter": 7, "formatOptions": { "linkTarget": "OpenBlade", "linkLabel": "Find in Firewall Log", "linkIsContextBlade": true, "bladeOpenContext": { "bladeName": "LogsBlade", "extensionName": "Microsoft_Azure_Monitoring_Logs", "bladeParameters": [ { "name": "resourceId", "source": "parameter", "value": "workspace" }, { "name": "query", "source": "cell", "value": "" } ] } } } ], "labelSettings": [ { "columnId": "transactionId_g", "label": "TransactionId" }, { "columnId": "access_log", "label": "Access Log" }, { "columnId": "firewall_log", "label": "Firewall Log" } ] } }, "customWidth": "25", "name": "query - 3" }, { "type": 3, "content": { "version": "KqlItem/1.0", "query": "AzureDiagnostics \r\n| where Category == \"ApplicationGatewayFirewallLog\"\r\n//| where TimeGenerated {detectionTime}\r\n| where transactionId_g == '{TransactionId}'\r\n| extend Id = new_guid()\r\n| extend ruleSetType_s = AdditionalFields.ruleSetType\r\n| extend ruleSetVersion_s = AdditionalFields.ruleSetVersion\r\n| extend ruleId_s = AdditionalFields.ruleId\r\n| extend hostname_s = AdditionalFields.hostname\r\n| extend details_message_s = AdditionalFields.details_message\r\n| extend details_file_s = AdditionalFields.details_file\r\n| extend details_line_s = AdditionalFields.details_line\r\n| project-reorder Id, action_s, ruleSetType_s, ruleSetVersion_s, ruleId_s, Message, hostname_s, requestUri_s, details_message_s, details_data_s, details_file_s, details_line_s, transactionId_g\r\n| extend normalizedRuleSetVersion = case(isempty(ruleSetVersion_s), \"\",\r\n // old engine: ruleSetVersion_s looks like \"3.1.0\" while type is \"OWASP_CRS\" => leave untouched, can use this as-is\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.1.0/rules/REQUEST-913-SCANNER-DETECTION.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+.[0-9]+\", ruleSetVersion_s,\r\n // new engine: ruleSetVersion_s looks like \"3.2\" while type is \"OWASP CRS\" => need a version that looks like \"3.2.0\"\r\n // final url to GH is like: https://github.com/coreruleset/coreruleset/blob/v3.2.0/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf#L56\r\n ruleSetVersion_s matches regex \"[0-9]+.[0-9]+\", strcat(ruleSetVersion_s, \".0\"),\r\n \"\")\r\n| extend normalizedDetailsFile = case( ruleSetType_s == \"OWASP_CRS\", replace(@'rules/', '', tostring(details_file_s)), // old engine: details_file_s looks like \"rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP_CRS\"\r\n ruleSetType_s == \"OWASP CRS\", details_file_s, // new engine: details_file_s looks like \"REQUEST-920-PROTOCOL-ENFORCEMENT.conf\" while type is \"OWASP CRS\"\r\n details_file_s)\r\n| extend Url = iif(isnotempty(ruleSetVersion_s) and ruleSetVersion_s !startswith \"1\", strcat('https://github.com/coreruleset/coreruleset/blob/', 'v', normalizedRuleSetVersion, '/rules/', normalizedDetailsFile, '#L', details_line_s), '')\r\n| project action_s, ruleSetType_s, ruleSetVersion_s, ruleId_s, Message, details_message_s, details_data_s, Url, TimeGenerated", "size": 0, "title": "Rules that got triggered for selected request", "exportedParameters": [ { "fieldName": "transactionId_g", "parameterName": "TransactionId" }, { "fieldName": "Message", "parameterName": "message", "parameterType": 1 }, { "fieldName": "details_message_s", "parameterName": "details", "parameterType": 1 }, { "fieldName": "details_data_s", "parameterName": "matcheddata", "parameterType": 1 }, { "fieldName": "Url", "parameterName": "SpiderLabsUrl", "parameterType": 1 } ], "queryType": 0, "resourceType": "microsoft.operationalinsights/workspaces", "crossComponentResources": [ "{workspace}" ], "visualization": "table", "gridSettings": { "formatters": [ { "columnMatch": "action_s", "formatter": 18, "formatOptions": { "thresholdsOptions": "icons", "thresholdsGrid": [ { "operator": "==", "thresholdValue": "Matched", "representation": "2", "text": "{0}{1}" }, { "operator": "==", "thresholdValue": "Blocked", "representation": "4", "text": "{0}{1}" }, { "operator": "Default", "thresholdValue": null, "representation": "more", "text": "{0}{1}" } ] } }, { "columnMatch": "ruleId_s", "formatter": 18, "formatOptions": { "thresholdsOptions": "colors", "thresholdsGrid": [ { "operator": "==", "thresholdValue": "{RuleId}", "representation": "yellow", "text": "{0}{1}" }, { "operator": "Default", "thresholdValue": null, "text": "{0}{1}" } ] } }, { "columnMatch": "Url", "formatter": 7, "formatOptions": { "linkTarget": "Url", "customColumnWidthSetting": "100%" }, "numberFormat": { "unit": 0, "options": { "style": "decimal" }, "emptyValCustomText": "No link for Custom or Bot rules" } }, { "columnMatch": "statusFromAction", "formatter": 5, "numberFormat": { "unit": 0, "options": { "style": "decimal" } } } ], "labelSettings": [ { "columnId": "action_s", "label": "Action" }, { "columnId": "ruleSetType_s", "label": "Rule Set Type" }, { "columnId": "ruleSetVersion_s", "label": "Rule Set Version" }, { "columnId": "ruleId_s", "label": "Rule Id" }, { "columnId": "Message", "label": "Message" }, { "columnId": "details_message_s", "label": "Details" }, { "columnId": "details_data_s", "label": "Matched Data" }, { "columnId": "TimeGenerated", "label": "Time Generated" } ] } }, "name": "TriggeredRules - By Url" }, { "type": 1, "content": { "json": "## Selection Summary:\r\n\r\nYou were looking for requests which triggered a WAF rule on url: _{RequestUri}_. \r\nYou have found an HTTP request, impacted by this rule, with transaction id equal to *{TransactionId}*. \r\n\r\nMore information on the selected rule which impacted this url, can be found on the [Core Rule Set Project](https://coreruleset.org) GitHub page, here: {SpiderLabsUrl}\r\n\r\n\r\n\r\nMore info on this request and the selected rule:\r\n- Message: {message}\r\n- Details: {details}\r\n- Matched Data: {matcheddata}" }, "name": "text - 7 - Copy" } ], "exportParameters": true }, "conditionalVisibility": { "parameterName": "SelectedTabPage", "comparison": "isEqualTo", "value": "triage-by-url" }, "name": "Triage by URL" } ], "defaultResourceIds": [ "/subscriptions/651dc44c-5d8e-48da-8cd3-cd79224ac290/resourceGroups/appgw-waf-triage/providers/Microsoft.OperationalInsights/workspaces/la-appgw-waf-triage" ], "$schema": "https://github.com/Microsoft/Application-Insights-Workbooks/blob/master/schema/workbook.json" } }, "resources": [ { "name": "[parameters('workbookId')]", "type": "microsoft.insights/workbooks", "location": "[resourceGroup().location]", "apiVersion": "2021-03-08", "dependsOn": [], "kind": "shared", "properties": { "displayName": "[parameters('workbookDisplayName')]", "serializedData": "[string(variables('workbookContent'))]", "version": "1.0", "sourceId": "[parameters('workbookSourceId')]", "category": "workbook" } } ], "outputs": { "workbookId": { "type": "string", "value": "[resourceId( 'microsoft.insights/workbooks', parameters('workbookId'))]" } }, "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#" }

shabaz-github commented

This issue was resolved by the user with the following PR - #343 - However user did not provide more details to include this in the repository - so closing this issue for now.