to improve indexing performance, we should let Elasticsearch do the update

Question

to improve indexing performance, we should let Elasticsearch do the update

ultrasaurus opened this issue 9 years ago · 2 comments

To implement nested documents, the current code fetches the whole document from Elasticsearch and then sends an update with a new document. This is highly inefficient, when es has the ability of updating a nested doc:

https://github.com/18F/open-data-maker/blob/dev/lib/data_magic/index.rb#L79-L85

Answer 1 · 2016-01-12T18:57:29.000Z

doc reference: https://www.elastic.co/guide/en/elasticsearch/guide/current/partial-updates.html

Answer 2 · 2016-02-18T20:56:21.000Z

I believe this is already implemented. When working on #295 I added a ES_DEBUG env variable and there is no ES fetch being performed. The ES update feature is already being used.