Lambda Timeouts on Ingest

[jbants]

I've set up the api following the documentation and new packages layout. I wanted to manually start the ingest process so I was following #52 and starting with Landsat. However, when I run the step function I am hitting the lambda timeout limit of 300 seconds. Has anyone else hit this?

The logs posted in #52 seem to indicate that the upload of files should be < 180 seconds.

I'm deploying in us-west-2 so maybe there is latency issues?

START RequestId: c908ae00-aa34-49c5-af3c-954c3c90dc69 Version: $LATEST
2018-07-28T03:52:16.280Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    ingest event: {"satellite":"landsat","arn":"arn:aws:states:us-west-2:126405998223:stateMachine:LandsatMetadataProcessorStateMachine-n78KcI1tLErq","maxFiles":8,"maxLambdas":10,"linesPerFile":300}
2018-07-28T03:52:27.028Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 250 files
2018-07-28T03:52:47.347Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 500 files
2018-07-28T03:53:08.545Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 750 files
2018-07-28T03:53:27.783Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 1000 files
2018-07-28T03:53:56.126Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 1250 files
2018-07-28T03:54:08.518Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 1500 files
2018-07-28T03:54:28.771Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 1750 files
2018-07-28T03:54:50.549Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 2000 files
2018-07-28T03:55:05.618Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 2250 files
2018-07-28T03:55:20.881Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 2500 files
2018-07-28T03:55:38.163Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 2750 files
2018-07-28T03:56:02.818Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 3000 files
2018-07-28T03:56:30.963Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 3250 files
2018-07-28T03:56:44.597Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 3500 files
2018-07-28T03:56:59.005Z    c908ae00-aa34-49c5-af3c-954c3c90dc69    uploaded 3750 files
END RequestId: c908ae00-aa34-49c5-af3c-954c3c90dc69
REPORT RequestId: c908ae00-aa34-49c5-af3c-954c3c90dc69  Duration: 300082.82 ms  Billed Duration: 300000 ms Memory Size: 1024 MB Max Memory Used: 297 MB
2018-07-28T03:57:16.361Z c908ae00-aa34-49c5-af3c-954c3c90dc69 Task timed out after 300.08 seconds

Any ideas or help is much appreciated!

[jbants]

Must have been a latency issue.
I rebuilt the api in us-east-1 and it grabbed the files without timing out.

[matthewhanson]

Glad it has worked out, I have run into timeouts before. If you consistently get timeout problems you can also reduce the linesPerFile parameter and it will split the CSV up into smaller files.

We're also working on having the ingestion be deployed as an ECS task using Fargate. This way it can ingest the entire file and not have to deal with splitting it up in order to run in Lambda functions.