codelibs/elasticsearch-dataformat

json format : remove line breaks in json source

fbaligand opened this issue · 2 comments

Problem :
Currently, if a document is stored in elasticsearch using "Kibana Dev Tools", it often contains "line breaks" (\r, \n).
And when we ask this plugin to generate JSON format, the line breaks are still present, which is an issue, because it breaks elasticsearch "bulk" format. And so, we can't directly use the result in a _bulk es query.

Wished Enhancement:
When chosen format is "json", could you remove Cariage Return (\r) and Line Feed (\n) characters in json source ?

Steps to reproduce the issue in Kibana Dev Tools:

# create a doc
PUT test1/test1/1
{
  "field1": "value1",
  "field2": "value2"
}

# export test1 index data in json format
POST test1/_data?format=json

# actual response
{"index":{"_index":"test1","_type":"test1","_id":"1"}}
{
  "field1": "value1",
  "field2": "value2"
}

# expected response
{"index":{"_index":"test1","_type":"test1","_id":"1"}}
{ "field1": "value1", "field2": "value2" }

# error when trying to import data using _bulk elasticsearch API
{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Malformed action/metadata line [3], expected START_OBJECT but found [VALUE_STRING]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Malformed action/metadata line [3], expected START_OBJECT but found [VALUE_STRING]"
  },
  "status": 400
}

Thanks @marevol for the fix !

I just tested it on Elasticsearch 6.1 and it works great !
Thanks for the backport on Elasticsearch 5.6, 6.0 and 6.1 !