Refactor Loki parse tree adjustments as a separate stage of backend
kelnage opened this issue · 0 comments
There are a number of changes that this code makes to the Sigma parse tree prior to converting it into a LogQL query. These include replacing strings that include wildcards with regular expressions, and removing NOT conditions through the application of De Morgan's law. At the moment this is done as the query string is being generated - but this choice has lead to multiple bugs (i.e., #13, #18) where expectations about the structure of the tree are incorrect or are not being applied uniformly throughout. It seems likely that as this code changes over time, more errors will crop up because of this (and our code will further diverge from the TextQueryBackend class, potentially leading to improvements to pySigma being incompatible with our own).
I propose the addition of a separate stage within convert_rule that makes relevant adjustments the parse tree, which will also allow us to simplify (if not remove completely) some of the custom conversion code. This could potentially be done as a pipeline, or even before the pipeline stage, although at this point, I can't identify any types of pipeline that rely on the parse tree structure, so this may not be necessary.