HTTPS problem

Question

HTTPS problem

subpublic opened this issue 5 years ago · 21 comments

Hi!

I've got the resizing working, but Lambda is not working on HTTPS.
I can reach my original (public) S3 image on
https://[bucket].s3.eu-north-1.amazonaws.com/[image].jpg and
http://[bucket].s3.eu-north-1.amazonaws.com/[image].jpg

And resized via Lambda on HTTP
http://[bucket].s3-website.eu-north-1.amazonaws.com/300x300/[image].jpg
But on on HTTPS
https://[bucket].s3-website.eu-north-1.amazonaws.com/300x300/[image].jpg
That just times out.

Though, when I have used Lambda over HTTP I can reach via HTTPS directly through S3.

Any ideas?

Answer 1 · 2019-10-10T12:22:50.000Z

Going to S3 -> Properties -> Static Website hosting
The "endpoint" is HTTP not HTTPS.

Answer 2 · 2019-10-27T10:13:42.000Z

Hi @subpublic,

Lambda works fine over HTTPS, either does S3 bucket.
The problem is in Static Web Hosting. Unfortunately,

The Amazon S3 website endpoints do not support HTTPS
https://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteHosting.html

There is an option, however, to use HTTPS. You can hook up your Domain. Then request a certificate.
You can google "aws static website hosting https", there are plenty of posts on how to do that.

At the moment, I can't find a simple solution. In case Amazon will support it, I will add the description about it.

Answer 3 · 2019-11-19T18:32:26.000Z

Hi. I tried adding HTTPS using this guide.
https://medium.com/@channaly/how-to-host-static-website-with-https-using-amazon-s3-251434490c59

Getting it to point to the S3 Bucket worked fine, but trying to point to the lambda static website (in CloudFront) did not go so well. My https domain just redirects to the http S3 static website instead.

Answer 4 · 2019-11-19T19:27:00.000Z

What have you set in your Lambda -> Environment variables?

Answer 5 · 2019-11-19T22:13:30.000Z

I have set these two variables

BUCKET = [my-bucket-name]
URL = http://[my-bucket-name].s3-website.eu-north-1.amazonaws.com

I tried using my new https domain instead

URL = https://static.mydomain.com

But that resulted in another problem, redirects to
https://[my-lambda].execute-api.eu-north-1.amazonaws.com/prod/aws-lambda-resize-dev?path=333xAUTO/image.jpg
With the error msg
[my-lambda].execute-api.eu-north-1.amazonaws.com redirected you too many times.

The action actually seems to work, as the new image gets generated and can be requested via my static https, but because it's redirected two times my https domain -> lambda -> back to my https domain I'm getting this error.

Answer 6 · 2019-11-20T09:13:58.000Z

Well, you definitely must have your (https) own domain set since it is your endpoint.
Once you get redirected back, you should be able to see your image in your browser.
If you get redirected back again instead, then one of the reasons might be that, although an image was generated, the previous request was cached hence the redirect.

Could you please look at this issue #5 and confirm wheather it is the same problem?

If you think you encounter the same problem, please, let me know and I will build the project under NodeJS 12x with caching problem fixed for you.

Answer 7 · 2019-11-20T10:11:51.000Z

Alright, could you check this version?
https://yadi.sk/d/rRXQULksMmMelg
It must not redirect you twice.

Answer 8 · 2019-11-20T12:31:22.000Z

I tried that, still same problem.
My first request to https domain gives

HTTP/2 307
content-length: 0
location: https://[my-lambda].execute-api.eu-north-1.amazonaws.com/prod/aws-lambda-resize-dev?path=624xAUTO/image.jpg

And requesting that responds

HTTP/2 301
date: Wed, 20 Nov 2019 12:24:25 GMT
content-type: application/json
content-length: 0
location: https://static.myserver.com/624xAUTO/image.jpg

If I after that go to https://static.myserver.com/624xAUTO/image.jpg it gives me the image.
The issue #5 writes something about changing TTL to 0, should I do that? (Didn't really understand where that was though)

Answer 9 · 2019-11-20T12:42:45.000Z

YAY! Changed the Default Cache TTL to 0, now everything works! (with the version you posted)

Answer 10 · 2019-11-20T13:42:26.000Z

I am really glad it helped. I should release the new version soon.

Now to prevent possible future problems like this, could you please share the steps you took?
In particular, you went to Lambda -> Function code -> Upload a .zip file and clicked on Upload.
Then what about Default Cache TTL to 0?
Have you done anything else?

Answer 11 · 2019-11-20T17:20:09.000Z

Yes

First i followed this guide to point https to my S3 website
https://medium.com/@channaly/how-to-host-static-website-with-https-using-amazon-s3-251434490c59

Then

I pointed the URL Variable to my https domain.
I uploaded the new zip, created a new release of Lambda and pointed "prod" to the new release.
Went to CloudFront -> chose my distribution -> Behaviours -> Edit. Here I could change Default TTL to 0.

Answer 12 · 2019-11-20T18:35:25.000Z

Well done @subpublic
Appreciate your explanation.

Answer 13 · 2019-12-16T14:38:18.000Z

Wanted to add I ran into issues when configuring this for my Cloud Front distribution.

On your distribution origin domain name make sure you enter the S3 web Endpoint - not the S3 origin.

Otherwise you will receive access denied errors. See issue and resolution here:
https://stackoverflow.com/questions/34060394/cloudfront-s3-website-the-specified-key-does-not-exist-when-an-implicit-ind/34065543#34065543

Answer 14 · 2020-01-29T23:05:46.000Z

Thanks @sagidM and @subpublic, this worked like a charm!!
Is it necessary to set the Default TTL to 0 in the CloudFront distribution behavior? I did a few tests without changing that value and it worked fine, so I don't quite understand why it needs to be changed to 0.

Answer 15 · 2020-03-14T00:54:51.000Z

I followed every step an I still get the Too many redirects error. Any clues?
It's crazy that there isn't an easier way to achieve this.

Answer 16 · 2020-03-14T12:59:54.000Z

@martinipsy this is not about HTTPS.
What the output can you see when you test your lambda? I.e.
{ "queryStringParameters": {"path": "my_directory_in_bucket/my_image.jpg"} }

Answer 17 · 2020-03-27T00:31:34.000Z

Dear Sagid, it has worked! I'm not sure what the issue was but I'm using it with the cloudfront URL. Thank you!

Answer 18 · 2020-04-14T09:53:20.000Z

Just to point out that without touching anything and having cloudfront setup, just having to point on the website (ex: xxx.s3-website.eu-west-3.amazonaws.com) did the trick.

I did not have to change the lambda function, the cache control, the default TTL, env variables.

Answer 19 · 2020-04-14T22:31:51.000Z

@GemN Appreciate your contribution. Some day I will add a detailed explanation of setting CloudFront up considering your comment.

Answer 20 · 2020-04-23T13:48:22.000Z

I am still getting ERR_TOO_MANY_REDIRECTS when I add an image such as https://images.mydomain.com/156x157/img.png as the src on an HTML image.

Current settings:
CloudFront Default Cache to 0.

Lambda
ENV variables:

BUCKET images.mydomain.com
URL https://images.mydomain.com

Route53
I have an A entry at images.mydomain.com at Route53 pointing to the CloudFront distribution.

Test
Web Browser
When I go to https://images.mydomain.com/156x157/img.png it works fine.

When I test the Lambda function with params

{
  "queryStringParameters": {
    "path": "156x156/img.png"
  }
}

I get this response:

{
  "statusCode": 301,
  "headers": {
    "Location": "https://images.sofadecasa.com/156x156/img.png"
  }
}

RELEASE
I am using s3-resizer_nodejs_12.13.0.zip

Any thoughts?

Answer 21 · 2020-04-23T22:01:51.000Z

I was able to make it work by point my src in the HTML to the CloudFront distribution https URL (make sure you select the default certificate, i.e., no need to generate a certificate for your domain).

I also changed the URL var in the Lambda function to point to the https URL provided by CloudFront. Everything works now.

If, like me, you are paranoid and don't wanna risk, checkout the onerror attribute of an image:
https://stackoverflow.com/questions/8124866/how-does-one-use-the-onerror-attribute-of-an-img-element