Bug with `NOT _exists_` query

[Technici4n]

Hi again, I am now having some issues with a NOT _exists_ query (again :P), this time from rule https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_susp_proc_wrong_parent.yml. This issue was introduced with the fix for #19. I added some line breaks in the Lucene query string to make the code more readable.

Output before the fix (v1.0.2):

(process.executable.lowercase:(
    *\\svchost.exe OR *\\taskhost.exe OR *\\lsm.exe OR *\\lsass.exe OR *\\services.exe OR *\\lsaiso.exe OR *\\csrss.exe OR *\\wininit.exe OR *\\winlogon.exe))
AND
(NOT (
        ((process.parent.executable.lowercase:(*\\SavService.exe OR *\\ngen.exe)) OR (process.parent.executable.lowercase:(*\\System32\\* OR *\\SysWOW64\\*)))
        OR
        ((process.parent.executable.lowercase:(*\\Windows\ Defender\\* OR *\\Microsoft\ Security\ Client\\*)) AND process.parent.executable.lowercase:*\\MsMpEng.exe)
        OR
        (NOT _exists_:process.parent.executable.lowercase OR process.parent.executable.lowercase:\-)
    )
)

Output after the fix (v1.0.3):

(process.executable.lowercase:(
    *\\svchost.exe OR *\\taskhost.exe OR *\\lsm.exe OR *\\lsass.exe OR *\\services.exe OR *\\lsaiso.exe OR *\\csrss.exe OR *\\wininit.exe OR *\\winlogon.exe))
AND
(NOT
    (
        ((process.parent.executable.lowercase:(*\\SavService.exe OR *\\ngen.exe)) OR (process.parent.executable.lowercase:(*\\System32\\* OR *\\SysWOW64\\*)))
        OR
        ((process.parent.executable.lowercase:(*\\Windows\ Defender\\* OR *\\Microsoft\ Security\ Client\\*)) AND process.parent.executable.lowercase:*\\MsMpEng.exe)
        OR
        (_exists_:process.parent.executable.lowercase OR process.parent.executable.lowercase:\-)
    )
)

Notice the missing NOT before _exists_.

Reproduction code:

from sigma.collection import SigmaCollection
from sigma.backends.elasticsearch import LuceneBackend
from sigma.pipelines.elasticsearch.windows import ecs_windows


raw_rule = r"""
title: Windows Processes Suspicious Parent Directory
id: 96036718-71cc-4027-a538-d1587e0006a7
status: test
description: Detect suspicious parent processes of well-known Windows processes
references:
    - https://securitybytes.io/blue-team-fundamentals-part-two-windows-processes-759fe15965e2
    - https://www.carbonblack.com/2014/06/10/screenshot-demo-hunt-evil-faster-than-ever-with-carbon-black/
    - https://www.13cubed.com/downloads/windows_process_genealogy_v2.pdf
author: vburov
date: 2019/02/23
modified: 2022/02/14
tags:
    - attack.defense_evasion
    - attack.t1036.003
    - attack.t1036.005
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        Image|endswith:
            - '\svchost.exe'
            - '\taskhost.exe'
            - '\lsm.exe'
            - '\lsass.exe'
            - '\services.exe'
            - '\lsaiso.exe'
            - '\csrss.exe'
            - '\wininit.exe'
            - '\winlogon.exe'
    filter_sys:
        - ParentImage|endswith:
            - '\SavService.exe'
            - '\ngen.exe'
        - ParentImage|contains:
            - '\System32\'
            - '\SysWOW64\'
    filter_msmpeng:
        ParentImage|contains:
            - '\Windows Defender\'
            - '\Microsoft Security Client\'
        ParentImage|endswith: '\MsMpEng.exe'
    filter_null:
        - ParentImage: null
        - ParentImage: '-'
    condition: selection and not 1 of filter_*
falsepositives:
    - Some security products seem to spawn these
level: low
"""


rules = SigmaCollection.from_yaml(raw_rule)
pipeline = ecs_windows()
backend = LuceneBackend(pipeline)
print(backend.convert(rules)[0])

Regards,

[Technici4n]

Hmmm, I believe the old output was not correct either. The following seems to work better¹:

(process.executable.lowercase:(
    *\\svchost.exe OR *\\taskhost.exe OR *\\lsm.exe OR *\\lsass.exe OR *\\services.exe OR *\\lsaiso.exe OR *\\csrss.exe OR *\\wininit.exe OR *\\winlogon.exe))
AND
(NOT (
        ((process.parent.executable.lowercase:(*\\SavService.exe OR *\\ngen.exe)) OR (process.parent.executable.lowercase:(*\\System32\\* OR *\\SysWOW64\\*)))
        OR
        ((process.parent.executable.lowercase:(*\\Windows\ Defender\\* OR *\\Microsoft\ Security\ Client\\*)) AND process.parent.executable.lowercase:*\\MsMpEng.exe)
        OR
        ((NOT _exists_:process.parent.executable.lowercase) OR process.parent.executable.lowercase:\-)
    )
)

Notice the extra parentheses around the NOT expression. Unfortunately I am not familiar enough with Lucene to understand why it needs that specific syntax, but I suspect always enclosing operators in parentheses would be a good idea?

Footnotes

1. I mean that it's required to correctly check that the field exists. ↩