jillesvangurp/kt-search

[FEAT] Allow routing to be set per item in a bulk request

Closed this issue · 2 comments

Describe the enhancement

Allow for a routing key to be passed in with each item in a bulk request. Currently routing can only be set for the whole bulk call

See Elasticsearch bulk API documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-routing

For example, currently I can only setting routing in the bulk parameters:

client.bulk(bulkSize = 50, routing = "123") {
  docs.forEach {
    index(source = objectMapper.writeValueAsString(it), index = indexName, id = it.id)
  }
}

However it can be useful to be able to set routing at an individual item level:

client.bulk(bulkSize = 50) {
  docs.forEach {
      index(source = objectMapper.writeValueAsString(it), index = indexName, id = it.id, routing = it.routing)
  }
}

Why is this needed?

To be able to bulk insert documents with different routing keys

How do you think it should be done?

Each of the functions in BulkSession should allow routing to be passed as an optional parameter.

In the case where a routing key is passed as a bulk parameter for the whole request and at the individual item level, then the individual item should take precedence. For example:

client.bulk(bulkSize = 50, routing = "123") {
  index(source = objectMapper.writeValueAsString(doc1), index = indexName, id = doc1.id)
  index(source = objectMapper.writeValueAsString(doc2), index = indexName, id = doc2.id, routing= doc2.routing)
}

In this case the first document would have a routing key = 123. Whereas the second document would have a routing key = to doc2.routing

Will you be able to help with a pull request?

I can have a go at getting a pull request for this

Sounds good. Let me know if you need any help.

fixed by #122