sat-utils/sat-api

Landsat lambda function ES client error

Closed this issue · 4 comments

Im trying to execute the landsat lambda function manually and am running into the following error:

{
    "errorMessage": "Cannot read property 'indices' of undefined",
    "errorType": "TypeError",
    "stackTrace": [
        "Object.putMapping (/var/task/index.js:364:30)",
        "Object.update (/var/task/index.js:758:13)",
        "satlib.es.client.then.then.then (/var/task/index.js:153750:34)",
        "<anonymous>",
        "process._tickDomainCallback (internal/process/next_tick.js:228:7)"
    ]
}

It seems that the function is being called three times (once for items, once for collections and once with client undefined.)
I added a check to see if client was undefined in the lambda function and it went past the above error, but hit a new one with:

{
    "errorMessage": "client is required",
    "errorType": "AssertionError [ERR_ASSERTION]",
    "stackTrace": [
        "new ElasticsearchWritable (/var/task/index.js:6936:5)",
        "Object.streamToEs (/var/task/index.js:442:20)",
        "processFile (/var/task/index.js:709:13)",
        "processFiles (/var/task/index.js:727:10)",
        "es.putMapping.then (/var/task/index.js:758:17)",
        "<anonymous>",
        "process._tickDomainCallback (internal/process/next_tick.js:228:7)"
    ]
}

This is a fresh deployment of the cloud formation file so oustide of the ingest files lambda function, nothing else has been run.
Any ideas?

hi @jbants , I assume this is with the latest changes on master and using the new deployment?

It could definitely be an issue with the new deployment. It worked when updating the existing site, but I will try and reproduce this on a brand new stack. Thanks for filing the issue!

@matthewhanson, That's correct. This is with the latest version on master and the new deployment.

My javascript is not super strong, but I'll keep digging and see what I come up with.
Please let me know if there is anything you would like me to test out!

Hi @matthewhanson, I have some more info.
I added a check for the ES client here:

const exist = await client.indices.exists({ index })
by using the client function on L:56.
client = await Client()

It seems to be getting a new client every time, rather than reusing the existing global client.

I'm getting a new client error from ES after making that change in the lambda code.

{
    "errorMessage": "client is required",
    "errorType": "AssertionError [ERR_ASSERTION]",
    "stackTrace": [
        "new ElasticsearchWritable (/var/task/index.js:6941:5)",
        "Object.streamToEs (/var/task/index.js:443:20)",
        "processFile (/var/task/index.js:714:13)",
        "processFiles (/var/task/index.js:732:10)",
        "es.putMapping.then (/var/task/index.js:763:17)",
        "<anonymous>",
        "process._tickDomainCallback (internal/process/next_tick.js:228:7)"
    ]
}

I'll keep digging, but wanted to post in case I get side tracked and forget to later on.

sorry for the delay @jbants , I've been on Safari in the Serengeti and thus off the grid.

When I get a chance I'll try a new deploy, also @scisco may be able to weigh in here as he refactored the code.