opencybersecurityalliance/stix-shifter

Improper "from_stix" mappings

pcoccoli opened this issue · 0 comments

pcoccoli commented

Describe the bug
Many connector module have from_stix_maps that map STIX references (e.g. email-message:from_ref) instead of mapping a property of the referenced object (in this example email-message:from_ref.value since from_ref references an email-addr object, and the actual email address is in that object's value property).

To Reproduce
Steps to reproduce the behavior:

  1. Try to use a valid STIX pattern:
$ python main.py translate gcp_chronicle query '{}' "[email-message:from_ref.value = 'mary@example.com']"
 2024-01-17 11:40:06,491 stix_shifter.stix_translation.stix_translation ERROR    Caught exception: Unable to map the following STIX objects and properties: ['email-message:from_ref.value'] to data source fields <class 'stix_shifter_utils.stix_translation.src.utils.exceptions.DataMappingException'>
 2024-01-17 11:40:06,491 stix_shifter_utils.stix_translation.stix_translation_error_mapper ERROR    received exception => DataMappingException: Unable to map the following STIX objects and properties: ['email-message:from_ref.value'] to data source fields
{
    "success": false,
    "connector": "gcp_chronicle",
    "code": "mapping_error",
    "error": "gcp_chronicle connector error => data mapping error : Unable to map the following STIX objects and properties: ['email-message:from_ref.value'] to data source fields"
}

This should work, since that's a perfectly valid STIX pattern and the target datasource has email data.

  1. Now try leaving out .value:
$ python main.py translate gcp_chronicle query '{}' "[email-message:from_ref = 'mary@example.com']"
 2024-01-17 11:41:42,748 stix_shifter_modules.gcp_chronicle.stix_translation.query_translator INFO     Converting STIX2 Pattern to data source query
 2024-01-17 11:41:42,748 stix_shifter_modules.gcp_chronicle.stix_translation.query_constructor INFO     GCP Chronicle Connector
{
    "queries": [
        {
            "ruleText": "rule cp4s_gcp_udi_rule_1705509702 { meta: author = \"ibm cp4s user\" description = \"Create event rule that should generate detections\" events: $udm.network.email.from = \"mary@example.com\" nocase condition: $udm}",
            "startTime": "2024-01-17T16:36:42.740Z",
            "endTime": "2024-01-17T16:41:42.740Z"
        }
    ]
}

This actually shouldn't work since the value from_ref in STIX would be a STIX object ID (e.g. "email-addr-" in 2.1 or just a number like "2" in 2.0).

Expected behavior
Valid STIX patterns, e.g. those that actually specify a STIX object path to a property (e.g. email-message:from_ref.value which points to a email-addr:value property), should work. Mapping a reference is weird, but I suppose stix-shifter could allow it (especially for backwards compatibility, since it has accepted it for a while).

List of all improper mappings

stix_shifter_modules/alertflex/stix_translation/json/from_stix_map.json:
network-traffic:src_ref
network-traffic:dst_ref

stix_shifter_modules/alertflex/stix_translation/json/stix_2_1/from_stix_map.json:
network-traffic:src_ref
network-traffic:dst_ref

stix_shifter_modules/azure_sentinel/stix_translation/json/stix_2_1/alert_from_stix_map.json:
x-ibm-finding:src_application_ref

stix_shifter_modules/azure_sentinel/stix_translation/json/alert_from_stix_map.json:
x-ibm-finding:src_application_ref

stix_shifter_modules/qradar/stix_translation/json/stix_2_1/events_from_stix_map.json:
file:parent_directory_ref

stix_shifter_modules/qradar/stix_translation/json/events_from_stix_map.json:
file:parent_directory_ref

stix_shifter_modules/crowdstrike/stix_translation/json/from_stix_map.json:
x-oca-asset:ip_refs

stix_shifter_modules/crowdstrike/stix_translation/json/stix_2_1/from_stix_map.json:
x-oca-asset:ip_refs

stix_shifter_modules/datadog/stix_translation/json/processes_from_stix_map.json:
process:creator_user_ref
process:parent_ref

stix_shifter_modules/ibm_security_verify/stix_translation/json/stix_2_1/from_stix_map.json:
x-oca-event:user_ref

stix_shifter_modules/ibm_security_verify/stix_translation/json/from_stix_map.json:
x-oca-event:user_ref

stix_shifter_modules/proofpoint/stix_translation/json/from_stix_map.json:
email-message:cc_refs
email-message:from_ref
email-message:to_refs
email-message:sender_ref
network-traffic:src_ref

stix_shifter_modules/qradar_perf_test/stix_translation/json/events_from_stix_map.json:
file:parent_directory_ref

stix_shifter_modules/qradar_perf_test/stix_translation/json/stix_2_1/events_from_stix_map.json:
file:parent_directory_ref

stix_shifter_modules/reaqta/stix_translation/json/stix_2_1/from_stix_map.json:
x-ibm-finding:ttp_tagging_refs

stix_shifter_modules/reaqta/stix_translation/json/from_stix_map.json:
x-ibm-finding:ttp_tagging_refs

stix_shifter_modules/sentinelone/stix_translation/json/stix_2_1/from_stix_map.json:
x-oca-event:host_ref
x-oca-event:url_ref
x-oca-event:file_ref
x-oca-event:process_ref
x-oca-event:parent_process_ref
x-oca-event:user_ref
x-oca-event:domain_ref
x-oca-event:registry_ref

stix_shifter_modules/sentinelone/stix_translation/json/from_stix_map.json:
x-oca-event:host_ref
x-oca-event:url_ref
x-oca-event:file_ref
x-oca-event:process_ref
x-oca-event:parent_process_ref
x-oca-event:user_ref
x-oca-event:domain_ref
x-oca-event:registry_ref

stix_shifter_modules/darktrace/stix_translation/json/from_stix_map.json:
x-oca-event:host_ref
x-oca-event:file_ref
x-oca-event:user_ref
x-oca-event:domain_ref

stix_shifter_modules/darktrace/stix_translation/json/stix_2_1/from_stix_map.json:
x-oca-event:host_ref
x-oca-event:file_ref
x-oca-event:user_ref
x-oca-event:domain_ref

stix_shifter_modules/gcp_chronicle/stix_translation/json/from_stix_map.json:
email-message:from_ref
x-ibm-finding:src_os_ref
x-ibm-finding:dst_os_ref

stix_shifter_modules/gcp_chronicle/stix_translation/json/stix_2_1/from_stix_map.json:
email-message:from_ref
x-ibm-finding:src_os_ref
x-ibm-finding:dst_os_ref

stix_shifter_modules/azure_log_analytics/stix_translation/json/stix_2_1/SecurityEvent_from_stix_map.json:
file:parent_directory_ref
x-ibm-finding:src_ip_ref
x-ibm-finding:src_application_user_ref
x-ibm-finding:dst_application_user_ref
x-ibm-finding:ioc_refs
x-oca-event:host_ref
x-oca-event:url_ref
x-oca-event:process_ref
x-oca-event:parent_process_ref
x-oca-event:user_ref
x-oca-event:x_service_file_ref

stix_shifter_modules/azure_log_analytics/stix_translation/json/stix_2_1/SecurityIncident_from_stix_map.json:
x-ibm-finding:x_owner_ref

stix_shifter_modules/azure_log_analytics/stix_translation/json/SecurityEvent_from_stix_map.json:
file:parent_directory_ref
x-ibm-finding:src_ip_ref
x-ibm-finding:src_application_user_ref
x-ibm-finding:dst_application_user_ref
x-ibm-finding:ioc_refs
x-oca-event:host_ref
x-oca-event:url_ref
x-oca-event:process_ref
x-oca-event:parent_process_ref
x-oca-event:user_ref
x-oca-event:x_service_file_ref

stix_shifter_modules/azure_log_analytics/stix_translation/json/SecurityIncident_from_stix_map.json:
x-ibm-finding:x_owner_ref

stix_shifter_modules/vectra/stix_translation/json/from_stix_map.json:
x-ibm-finding:src_ip_ref

stix_shifter_modules/vectra/stix_translation/json/stix_2_1/from_stix_map.json:
x-ibm-finding:src_ip_ref

stix_shifter_modules/cisco_secure_email/stix_translation/json/from_stix_map.json:
email-message:from_ref
email-message:sender_ref
email-message:to_refs
email-message:x_sender_ip_ref

stix_shifter_modules/cisco_secure_email/stix_translation/json/stix_2_1/from_stix_map.json:
email-message:from_ref
email-message:sender_ref
email-message:to_refs
email-message:x_sender_ip_ref