logstash-plugins/logstash-filter-fingerprint

Implement ECS-Compatibility Mode

yaauie opened this issue · 3 comments

As a part of the effort to make plugins able to run in an ECS-Compatible manner
by default in an upcoming release of Logstash, this plugin needs to implement
an ECS-Compatibility mode that does not implicitly use fields that conflict
with ECS.

Absent target directive, this plugin uses root-level field fingerprint,
which is undefined in ECS and therefore has the possiblity of conflicting
with future versions of ECS. The plugin may benefit from an ECS-Compatibility
mode that requires the target to be set explicitly or otherwise reduces the
risk of future conflict.

kares commented

would we also consider, at this stage, changing (default) target => "event.hash" in ECS mode?

🤔 If we set event.hash, we should do so on a named subfield for the algorithm used in a way that aligns with the ECS Hash type (e.g., when method => SHA512, the sensible default is event.hash.sha_512).

> Field names for common hashes (e.g. MD5, SHA1) are predefined. Add fields for other hashes by lowercasing the hash algorithm name and using underscore separators as appropriate (snake case, e.g. sha3_512).

This gets a little tricky when the plugin is configured to use a non-standard algorithms, but IMHO these should be deprecated and implemented elsewhere anyway:

  • IPV4_NETWORK (not a fingerprint at all)
  • PUNCTUATION (effectively a gsub stripping punctuation?)
  • UUID (non-deterministic by design)

EDIT: event.hash is keyword type, not a Hash type.

I'm +1 to defaulting to target => "[event][hash]" in ECS-Compatibility mode.