Add functionality to read list of fields to hash from the event itself

[soenkeliebau]

We are currently hashing privacy sensitive data, but want to give control over what is security sensitive to the data sender, not have this configuration fixed in Logstash.

To achieve this, we have implemented a ruby filter to read a list of fields that should be hashed from a specific field in the log event.

So for example we could have an event of this format:

{ "maskingfields":["name", "phone"], "phone":"123", "name":"secretname", "action":"post", ...}

and in this case the fields name and phone would be hashed, everything else left as is.

To allow for multiple fields to be anonymized we configure a prefix (for example datamasking.fieldname) that is added to the fieldname to obtain the target fieldname to keep the original value before applying the hash algorithm.

So for example the cleartext value of name would end up in datamasking.name and name would contain the hashed value. This allows us to define field level security roles and exclude users from seeing all fields prefixed with datamasking. but keeps the ability to allow certain users access to the cleartext data.

If people are interested in this I'd like to contribute this to the filter, but would like some input on how a sensible and backwards compatible configuration extension for this might look like.