memgraph/mage

[BUG] Elasticsearch synchronization is not always able to read in data from elastic

Closed this issue · 3 comments

Describe the bug
On Elastic 8.14.3, reading docs from elastic with search() or scan() fails with a Python KeyError of "index" not found

To Reproduce
Steps to reproduce the behavior:

  1. Connect to an 8.14.3 elastic instance from memgraph Cypher editor
  2. In memgraph Cypher editor, run a query to get data from an index in that elastic index
CALL elastic_search_serialization.search("<some index in the elastic instance>",  "{\"match_all\": {}}", 1000, 0)
YIELD *
RETURN *;

Expected behavior
Return a valid response from the elasticsearch query without error

Additional context
Add any other context about the problem here.
Fix is straightforward. In the two functions search() and scan() in elastic_search_serialization.py, there is a code block reading (in search)

    for hit in response[HITS][HITS]:
        hit[ID] = hit[_SOURCE][INDEX][ID]
        hit[_SOURCE].pop(INDEX, None)
        hits.append(hit)

and (in scan)

    for item in response:
        item[ID] = item[_SOURCE][INDEX][ID]
        item[_SOURCE].pop(INDEX, None)
        items.append(item)

These blocks can be updated to (in search)

    for hit in response[HITS][HITS]:
        if hit[_SOURCE].get(INDEX, None):
            hit[ID] = hit[_SOURCE][INDEX][ID]
            hit[_SOURCE].pop(INDEX, None)
        hits.append(hit)

and (in scan)

    for item in response:
        if item[_SOURCE].get(INDEX, None):
            item[ID] = item[_SOURCE][INDEX][ID]
            item[_SOURCE].pop(INDEX, None)
        items.append(item)

Thanks a lot @nagi49000 for reporting!
Since the fix is simple, feel free to post a PR 🙏
Hopefully, the team can pick this up quickly 💪

Thanks @gitbuda - PR up #505

Fixed by #505