[BUG] Elasticsearch synchronization is not always able to read in data from elastic
Closed this issue · 3 comments
nagi49000 commented
Describe the bug
On Elastic 8.14.3, reading docs from elastic with search() or scan() fails with a Python KeyError of "index" not found
To Reproduce
Steps to reproduce the behavior:
- Connect to an 8.14.3 elastic instance from memgraph Cypher editor
- In memgraph Cypher editor, run a query to get data from an index in that elastic index
CALL elastic_search_serialization.search("<some index in the elastic instance>", "{\"match_all\": {}}", 1000, 0)
YIELD *
RETURN *;
Expected behavior
Return a valid response from the elasticsearch query without error
Additional context
Add any other context about the problem here.
Fix is straightforward. In the two functions search()
and scan()
in elastic_search_serialization.py
, there is a code block reading (in search)
for hit in response[HITS][HITS]:
hit[ID] = hit[_SOURCE][INDEX][ID]
hit[_SOURCE].pop(INDEX, None)
hits.append(hit)
and (in scan)
for item in response:
item[ID] = item[_SOURCE][INDEX][ID]
item[_SOURCE].pop(INDEX, None)
items.append(item)
These blocks can be updated to (in search)
for hit in response[HITS][HITS]:
if hit[_SOURCE].get(INDEX, None):
hit[ID] = hit[_SOURCE][INDEX][ID]
hit[_SOURCE].pop(INDEX, None)
hits.append(hit)
and (in scan)
for item in response:
if item[_SOURCE].get(INDEX, None):
item[ID] = item[_SOURCE][INDEX][ID]
item[_SOURCE].pop(INDEX, None)
items.append(item)
gitbuda commented
Thanks a lot @nagi49000 for reporting!
Since the fix is simple, feel free to post a PR 🙏
Hopefully, the team can pick this up quickly 💪