Allow for top-level Elasticsearch `mapping` to be defined

[jvanderen1]

It would be nice to define an explicit Elasticsearch mapping (minus any _meta fields` at the top of the schema. Something akin to what is in the Elasticsearch documentation:

PUT /my-index-000001
{
  "mappings": {
    "properties": {
      "age":    { "type": "integer" },  
      "email":  { "type": "keyword"  }, 
      "name":   { "type": "text"  }     
    }
  }
}

[nowfred]

I believe you can do this by setting any mapping you want inside transform.mapping: {} in the schema.json file. We use this to create abstract mappings that apply to some downstream transforms - working well so far.

[jvanderen1]

@nowfred The problem is that we use custom plugins to shape the schema into a custom format before indexing. Our mappings don't necessarily align with PGSync's schema output.

We need a way to define a custom Elasticsearch mappings without referencing columns.

[toluaina]

I have added support for this in the master branch. Please give it a try. Here is a schema example

[jvanderen1]

@toluaina That's exciting! I will give it a shot 👍🏻

[jvanderen1]

@toluaina Unfortunately, this does not seem to work for me. I added the following field:

[
  {
    ...
    "mappings": {
      "properties": {
        "name": {
          "type": "text",
          "fields": {
            "raw": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    }
  }
]

I receive the following error:

elasticsearch.BadRequestError: BadRequestError(400, 'mapper_parsing_exception', 'Root mapping definition has unsupported parameters:  [index : {properties={collection={type=text, fields={raw={ignore_above=256, type=keyword}}}}}]')