mallocator/Elasticsearch-Exporter

Importing from File does not work

chrisspiegl opened this issue · 3 comments

I am experiencing some problems with importing uncompressed data from disk.

I deleted the index I want to write into before I ran this. Am I missing something obvious? Please help.

The Output:

Elasticsearch Exporter - Version 1.4.0
Reading mapping from meta file database1.meta
Waiting for mapping on target host to be ready, queue length 5
Creating type mapping in target ElasticSearch instance
Mapping is now ready. Starting with 5 queued hits.
Host es.amazine.vm:9200 responded to PUT request on endpoint /test/_mapping/story/ with an error:
 {"error":"MapperParsingException[Root type mapping not empty after parsing! Remaining fields:   [test : {mappings={story={properties={title={type=string}, tags={type=string, fields={type=completion, index_analyzer=simple, search_analyzer=simple, preserve_separators=false, payloads=false}}}}}}]]","status":400}
Number of calls:    1
Fetched Entries:    5 documents
Processed Entries:  5 documents
Source DB Size:     5 documents
Peak Memory Used:   5700288 bytes (50%)
Total Memory:       11344896 bytes

What version of Elasticsearch are you using?

Can you show me the content of the .meta file that you're importing form?

As a workaround you can always first create the index with the correct mapping and then run the exporter, which will not overwrite existing mappings.

UPDATE

I just found out that the problem was in my data. I changed the exported data a bit and there had been errors then.

There are only two things that persist:

The error Host es.amazine.vm:9200 responded to PUT request on endpoint /test/_mapping/story/ with an error: {"error":"MapperParsingException[Root type mapping not empty after parsing! Remaining fields: [test : {mappings={story={properties={title={type=string}, tags={type=string, fields={type=completion, index_analyzer=simple, search_analyzer=simple, preserve_separators=false, payloads=false}}}}}}]]","status":400} is still written to the screen.

And the mapping that gets created is not the once in the .meta file (see old comment).

But with manually creating the index beforehand it works. Sorry for bothering you.

OLD Comment

The ES version is: 1.4.0
The .meta content is at the end of the message. The weird thing is if I do the -N true it kinda works just setting the mapping. But it sets the mapping as follows:

The Mapping That Actually Got Created

{
  "story": {
    "properties": {
      "tags": {
        "type": "string"
      },
      "title": {
        "type": "string"
      }
    }
  }
}

Content of the .meta File

{
  "test": {
    "mappings": {
      "story": {
        "properties": {
          "title": {
            "type": "string"
          },
          "tags": {
            "type": "string",
            "fields": {
              "type": "completion",
              "index_analyzer": "simple",
              "search_analyzer": "simple",
              "preserve_separators": false,
              "payloads": false
            }
          }
        }
      }
    }
  },
  "_scope": "type",
  "_index": "test",
  "_type": "story"
}

P.S.: if I run do the mapping manually, it actually sets the mapping the way I expect it to. If I then try to run the import of my data it gives the same error as above (first issue message). I am really kinda confused since the first 5 elements are actually imported.

Hey no bother. I'm happy to help if I can, and I appreciate any feedback that I get.