ulule/python-logstash-formatter

Update to the latest Logstash Schema

Closed this issue · 3 comments

Hello,

Logstash has updated their spec for schema:

https://logstash.jira.com/browse/LOGSTASH-675

{
  "@timestamp": "2012-12-18T01:01:46.092538Z".
  "@version": 1,
}

those are the only two fields that need an "@" sign and @fields is no longer required.

ie:

{
  "account": "pyr",
  "args": [],
  "created": 1367480388.013037,
  "exception": [
    "Traceback (most recent call last):\n",
    "  File \"toto.py\", line 16, in <module>\n    k['unknown']\n",
    "KeyError: 'unknown'\n"
  ],
  "filename": "toto.py",
  "funcName": "<module>",
  "levelname": "WARNING",
  "levelno": 30,
  "lineno": 18,
  "module": "toto",
  "msecs": 13.036966323852539,
  "name": "root",
  "pathname": "toto.py",
  "process": 1819,
  "processName": "MainProcess",
  "relativeCreated": 18.002986907958984,
  "thread": 140060726359808,
  "threadName": "MainThread"
  "message": "TOTO",
  "source_host": "phoenix.spootnik.org",
  "@timestamp": "2013-05-02T09:39:48.013158",
  "@version": 1
}
pyr commented

@joekiller yup, i've been meaning to update this. are you working on a PR for this ?

@pyr check out the pull request.

This basically converts something like

{
    "@fields": {
        "relativeCreated": 6251.781940460205,
        "process": 5342,
        "args": [],
        "module": "go",
        "funcName": "kill_poller",
        "name": "run-program",
        "thread": 140735190438272,
        "created": 1382039320.24001,
        "threadName": "MainThread",
        "msecs": 240.01002311706543,
        "filename": "go.py",
        "levelno": 20,
        "processName": "MainProcess",
        "pathname": "/path/to/gun.py",
        "lineno": 177,
        "levelname": "INFO"
    },
    "@timestamp": "2013-10-17T19:48:40.240067Z",
    "@source_host": "localhost.local",
    "@message": ""
}

to

{
    "relativeCreated": 13899.285078048706,
    "process": 5325,
    "@timestamp": "2013-10-17T19:47:04.229860Z",
    "args": [],
    "module": "go",
    "funcName": "start_poller",
    "name": "run-program",
    "thread": 4395577344,
    "created": 1382039224.229643,
    "threadName": "Gun",
    "msecs": 229.6431064605713,
    "filename": "go.py",
    "levelno": 20,
    "processName": "MainProcess",
    "source_host": "localhost.local",
    "pathname": "/path/to/go.py",
    "lineno": 392,
    "@version": 1,
    "levelname": "INFO"
}