Undeprecate Pipe Operator

[andrewthad]

The specification describes the pipe operator (search_expression | aggregation_expression) as deprecated. However, there is no alternative to it, and it is still used in lots of sigma rules. Deprecated features usually have an alternative that they can be replaced with, and so I think it is misleading to say that it is deprecated. I think it would be more clear to say something like "we are planning to replace this with something else, but it's not ready yet". I've written a sigma integration recently and was confused by this. For me, different language would have communicated more clearly that implementers need to support the pipe operator.

[nasbench]

Hi,

The SIGMA pipe operator is deprecated and it's stated in the specs that it'll be replaced by the Sigma correlations [See below]

Sigma correlations/meta-rules are described in a separate branch V2 here is a link to it https://github.com/SigmaHQ/sigma-specification/blob/version_2/Sigma_meta_rules.md

All SIGMA rules using that operator will be deprecated and replaced with the new notation. It's now discouraged to write rules using that old notation :)

[andrewthad]

But it hasn't been replaced by Sigma correlations yet. I see the documentation for Sigma correlations, but it's not clear to me that it is actually used in Sigma. For example, if I run

find . -name 'mr_correlation_*'

in the sigma project, I don't see any rules that use correlations. Put differently, if I rewrite an existing rule to use correlations instead and PR that to Sigma, will that be accepted? If not, then describing the pipe operator as deprecated feels strange because there would not be a path forward for anyone trying to replace uses of it.

[nasbench]

I don't know what you're looking for with that search. But SIGMA correlation is still in dev and the spec is not final. The deprecation of the correlation notation is stating that the current state of it deprecated meaning that it's not in dev and will not be maintained similarly to the whole of SIGMAC (only best effort and major bugs).

As for rules. We're not accepting any new rules using that old notation since we're deprecating SIGMAC and since the new notation, it's still in dev you can't use it either in the main repo.

Since we're easing the transition to PySIGMA , currently, only new modifiers introduced with PySIGMA are accepted for new rules. So new rules using modifiers such as windash, cidr, expand...etc are good to go.

[andrewthad]

Thanks for clarifying that. I'm going to go ahead and close this since I have the answers to the questions I was wondering, but I think it is worthwhile to state explicitly somewhere (other than the issue tracker) that at the moment, correlated rules cannot be added to SIGMA. I would not have been able to figure that out without your explanation. Perhaps something else that would help communicate this would be to deprecate all rules in SIGMA that use the pipe operator. Most of these are marked as test or experimental, but a rule using a deprecated feature should probably be considered deprecated itself.