[FEAT] Allow routing to be set per item in a bulk request
Closed this issue · 2 comments
Describe the enhancement
Allow for a routing key to be passed in with each item in a bulk request. Currently routing can only be set for the whole bulk call
See Elasticsearch bulk API documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-routing
For example, currently I can only setting routing in the bulk parameters:
client.bulk(bulkSize = 50, routing = "123") {
docs.forEach {
index(source = objectMapper.writeValueAsString(it), index = indexName, id = it.id)
}
}
However it can be useful to be able to set routing at an individual item level:
client.bulk(bulkSize = 50) {
docs.forEach {
index(source = objectMapper.writeValueAsString(it), index = indexName, id = it.id, routing = it.routing)
}
}
Why is this needed?
To be able to bulk insert documents with different routing keys
How do you think it should be done?
Each of the functions in BulkSession
should allow routing to be passed as an optional parameter.
In the case where a routing key is passed as a bulk parameter for the whole request and at the individual item level, then the individual item should take precedence. For example:
client.bulk(bulkSize = 50, routing = "123") {
index(source = objectMapper.writeValueAsString(doc1), index = indexName, id = doc1.id)
index(source = objectMapper.writeValueAsString(doc2), index = indexName, id = doc2.id, routing= doc2.routing)
}
In this case the first document would have a routing key = 123. Whereas the second document would have a routing key = to doc2.routing
Will you be able to help with a pull request?
I can have a go at getting a pull request for this
Sounds good. Let me know if you need any help.
fixed by #122