parse-nl/SystemdJournal2Gelf

remove JSON parsing as graylog now supports it fully

SjonHortensius opened this issue · 2 comments

remove json parsing from SystemdJournal2Gelf - let users configure this themselves as a pipeline. Steps to recreate this as a custom pipeline:

  • Go to /system/pipelines in your graylog install and create a new pipeline

  • Edit the new pipeline and add a stage:

  • In the first stage, add a rule to interpret the message as json:

rule "interpret message as JSON"
when
    has_field("message") && starts_with(to_string($message.message), "{\"")
then
    let json = parse_json(to_string($message.message));
    let map = to_map(json);
    set_fields(map);

    rename_field("Message", "message");
    rename_field("FullMessage", "full_message");
end
  • you can prefix additional stages to cleanup your message eg. when receiving messages from php-fpm:
rule "strip fpm pool prefix"
when
  has_field("message") && starts_with(to_string($message.message), "pool ")
then
  set_field("message", regex_replace("^pool [a-zA-Z_\\[\\d\\]]+: ", to_string($message.message), ""));
end
  • attach the pipeline to the appropriate stream(s). Make sure the Pipeline Processor is configured after Message Filter Chain in system/configurations

Hi @SjonHortensius just found this by accident.
I think the README.md requires updating, so people don't expect this JSON parsing to happen automatically.

@hmmmsausages you're right - I've updated the README.MD