amazon-archives/serverless-image-resizing

error with the example: The specified key does not exist.

Closed this issue · 18 comments

Andriy-Kulak commented

I followed the instructions on the AWS blog to set this project up. Upon testing in the end, I get the following error.

<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>200x200/earthSearch.jpg</Key>

This is the same type of error that you would get if the image doesn't exist in S3 bucket which leads me to assume that something is wrong with my redirect rules, which is odd because it seems like a simple step. I added the rules in the right place per screenshot below

My prefixes are the same as yours. The only thing I can imagine is different is that I couldn't add dashes inside the HostName tag (it wouldn't allow me to save with daashes). So to follow your hello world example, my hostname looked something like the code below.

My API endpoint is live too and hooked up to lambda func. Anything else I am missing?

Thanks for putting this together!

      <HostName>h3ll0w0rld.execute-api.us-east-1.amazonaws.com</HostName>
      <ReplaceKeyPrefixWith>prod/resize?key=</ReplaceKeyPrefixWith>

screen shot 2017-03-29 at 9 40 29 pm

jpignata commented

Hi! Can you provide your complete redirection rules XML document?

Andriy-Kulak commented

@jpignata see below. also, i tried doing the imgae tests with http:// and https:// prefixes which didn't make a difference. I am using your hello world example in host not to expose mine to everyone since I will be using it in the future.

<RoutingRules>
  <RoutingRule>
    <Condition>
      <KeyPrefixEquals/>
      <HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
    </Condition>
    <Redirect>
      <Protocol>https</Protocol>
      <HostName>h3ll0w0rld.execute-api.us-east-1.amazonaws.com</HostName>
      <ReplaceKeyPrefixWith>prod/resize?key=</ReplaceKeyPrefixWith>
      <HttpRedirectCode>307</HttpRedirectCode>
    </Redirect>
  </RoutingRule>
</RoutingRules>
jpignata commented

Hmm - the h3ll0w0rld looks like the one from the example, so you'll need to enter your actual API host in there. Still, I'd expect that rule to redirect you somewhere that doesn't exist rather than returning a 404.

I'm guessing you're not hitting the website endpoint of your bucket. Can you use that instead? You can find it in the Static Website Hosting settings.

michabbb commented

@jpignata i followed the howto step by step, same problem here, i also get a 404 and btw when i try to call the API/Lambda function directly, i get this error: {"message": "Internal server error"}
and yes, i really used the correct URLs like you explained in the howto.
when testing the lambda function with - don´t know if that is the correct test "Rekognition S3 Request" and when i update in there the values for my bucket (name, arn, etag, key) i get the error:

{
  "errorMessage": "Module version mismatch. Expected 48, got 46.",
  "errorType": "Error",
  "stackTrace": [
    "Function.Module._load (module.js:438:3)",
    "Module.require (module.js:497:17)",
    "require (internal/module.js:20:19)",
    "Object.<anonymous> (/var/task/node_modules/sharp/index.js:12:13)",
    "Module._compile (module.js:570:32)",
    "Object.Module._extensions..js (module.js:579:10)",
    "Module.load (module.js:487:32)",
    "tryModuleLoad (module.js:446:12)",
    "Function.Module._load (module.js:438:3)",
    "Module.require (module.js:497:17)",
    "require (internal/module.js:20:19)",
    "Object.<anonymous> (/var/task/index.js:3:13)"
  ]
}

and btw.. the internal error does not appear in the cloudwatch ;-(

michabbb commented

@jpignata ARG in your blog posting you didn´t mention

Note: If you create the Lambda function yourself, make sure to select Node.js version 4.3.

i will check this, because this seems to be the reason for the module mismatch problem....

UPDATE: no, still get the error ;-((

how can i test the lambda function inside the console ??? howto simulate a get call with the key ?

UPDATE:

i see new cloudwatch errors, but don´t know, what they mean:

{
    "errorMessage": "The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.",
    "errorType": "InvalidRequest",
    "stackTrace": [
        "Request.extractError (/var/runtime/node_modules/aws-sdk/lib/services/s3.js:539:35)",
        "Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)",
        "Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)",
        "Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:671:14)",
        "Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)",
        "AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)",
        "/var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10",
        "Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)",
        "Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:673:12)",
        "Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)",
        "Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)",
        "Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:671:14)",
        "Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)",
        "AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)",
        "/var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10",
        "Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)"
    ]
}

UPDATE: what a journey.... got here to there... and now will check if

var S3 = new AWS.S3({signatureVersion: 'v4'});

will to the trick, because i use Frankfurt as location for my s3 bucket....

jpignata commented
michabbb commented

@jpignata please help, i get mad with this... now i get this error:

Unable to import module 'index': Error at Function.Module._resolveFilename (module.js:325:15) at Function.Module._load (module.js:276:25) at Module.require (module.js:353:17) at require (internal/module.js:12:17)
Unable to import module 'index': Error
at Function.Module._resolveFilename (module.js:325:15)
at Function.Module._load (module.js:276:25)
at Module.require (module.js:353:17)
at require (internal/module.js:12:17)

what are the correct settinsg here ???

schnappschuss_033017_011817_pm

jpignata commented
michabbb commented

@jpignata many thanks, i zipped the folder, not the files inside ;-(

okay, i am close to it: i had to set the timeout > 10 because 3 seconds are not enough, you should mention that if you work with very large source images ;)

now i can call the lambda url directly and get redirected to the new resized image... nice....

but, the redirect of the bucket still does not work ;-((

still get this error:

<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>100x100/100004_1600x1600_v1.png</Key>
<RequestId>xxxxxxxxxxx</RequestId>
<HostId>
xxxxxxxxxxxxx
</HostId>
</Error>

schnappschuss_033017_014520_pm

my routing rule:

<RoutingRules>
  <RoutingRule>
    <Condition>
      <KeyPrefixEquals/>
      <HttpErrorCodeReturnedEquals>404</HttpErrorCodeReturnedEquals>
    </Condition>
    <Redirect>
      <Protocol>https</Protocol>
      <HostName>xxxxxxxxxxx.execute-api.eu-west-1.amazonaws.com</HostName>
      <ReplaceKeyPrefixWith>prod/resize-image?key=</ReplaceKeyPrefixWith>
      <HttpRedirectCode>307</HttpRedirectCode>
    </Redirect>
  </RoutingRule>
</RoutingRules>

yes, the lambda is called "resize-image", thats not the problem ;)

jpignata commented
michabbb commented

@jpignata can i share something with you in private ???

jpignata commented
michabbb commented

@jpignata you got mail, thanks ;)

jpignata commented

@Andriy-Kulak I'm going to close this issue but please feel free to re-open if my advice to ensure you're using the bucket's Static Website Endpoint isn't working. Feel free to email me as well.

Thanks!

Andriy-Kulak commented

@jpignata Thanks for all the help so far! I was able to get to the static part per your recommendation and am now getting redirected but I got another error, this time auth related. I sent you an email with the specifics and will provide an update here once I am able to resolve it (in case anyone else will face it in the future).

michabbb commented

@jpignata I guess other come to the same idea like me, using cloudfront, but that brings new problems

  1. you have to use the static website URL instead of the name of your bucket from the drop down (when you create the origin), to make redirects working

  2. that's my main problem: cloudfront caches the result. So what happens?

  • you do the request via cloudfront url
  • you get redirected to lambda, lambda creates
    The image and redirects you back to the cloudfront url (because I changed to url to cloudfront, of course)
  • cloudfront says: ups, I already know what to do without asking s3, I sent you back to lambda

So you get an endless loop 😕

Any idea how to solve this?

Thanks!

jpignata commented

@michabbb You'll have to configure CloudFront not to cache the result. See the documentation for detials.

  • 👍1
tomgallagher commented

Hello

Can anyone tell me how this was resolved? I'm also getting

<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>

and the function is not getting invoked.