bobbui/json-logging-python

Allow changing build in fields names

OranShuster opened this issue · 1 comments

My use case is that for any log we expect it to have a @timestamp field in the root
It can come from either

  1. Filebeat putting it's ingestion time there
  2. Our different pipelines parsing non-json logs and putting the correct value there
  3. Filebeat/logstash/ingest pipelines processing json logs

So in order to make the default format work with that flow i wrote a simple formatter that uses all the base classes formatting because i want those "for free"

class ELKJsonLogFormatter(json_logging.JSONLogFormatter):
    def format(self, record):
        log = super().format(record=record)
        log_as_dict = json.loads(log)
        log_as_dict["@timestamp"] = log_as_dict["written_ts"]
        modified_log = json.dumps(log_as_dict)
        return modified_log

While this is not horrible, i feel like the loads -> dumps is a total waste for just copying 1 field to the other

@OranShuster u can extend the same class or BaseJSONFormatter if u would like more customization. just extend the _format_log_object similar to what we have here: https://github.com/bobbui/json-logging-python/blob/master/json_logging/__init__.py#L322