auto-ssl/lua-resty-auto-ssl

Deployment using ECS behind a Network Load Balancer

rhlr opened this issue · 5 comments

rhlr commented

I'm trying to deploy the Docker on AWS ECS behind a Network Load Balancer as I need a static IP. The deployment is successful but I'm facing one issue. When nginx redirects to https , I get error ERR_SSL_PROTOCOL_ERROR . I figured out that this might be because I'm listening to a TCP port while communication with TLS. Now, if I change the Load Balancer's listener to TLS and the target group to another TLS listening to 443, it passes the request but the problem is for TLS to work as a listener, I need a SSL certificate which overrides the certificate generated by auto-ssl. Is there any workaround this ?

christian-s commented

Hi @rhlr,
TCP is correct. If you switch to TLS, then the NLB does the SSL termination. This is exactly not what you want.
ERR_SSL_PROTOCOL_ERROR means that the browser was able to connect but unable to establish a SSL connection for some reason.
Typical error is that you connected your NLB Port 443 to the wrong port on your container or forgot to open that port in the security group.
Another thing is that this works only with a single container, this is because you have two target groups, one for each port. The https request might be answered by a different container than the http request which breaks your session with letsencrypt.

  • 👍1
rhlr commented

@christian-s Thanks for your suggestion. This was indeed the error. I created two separate target groups one listening at 80 and another one at 443 and it started working.
Also, I needed to implement auto scaling in the container, but then the same problem of session will arise as you have mentioned. I was thinking of blocking port 80 and redirecting everything to 443 as session stickiness is not yet supported by NLB. Maybe the nginx block will change to something like this:

  # HTTPS server
    server {
      listen 443 ssl;
      server_name _;
      location /.well-known/acme-challenge/ {
        content_by_lua_block {
          auto_ssl:challenge_server()
         }
       }
       ssl_certificate_by_lua_block {
            auto_ssl:ssl_certificate()
        }
        ssl_certificate /etc/ssl/resty-auto-ssl-fallback.crt;
        ssl_certificate_key /etc/ssl/resty-auto-ssl-fallback.key;
     }

But I'm not sure if the acme-challenge route will work on port 443. I will give it a try and check.

christian-s commented

This will not work. You need to integrate redis to achieve what you want. Another recommendation is to separate the SSL offloading from your backend containers as an own service to scale and update them separately.
If you can accept a lower availability, just run your lua-resty-auto-ssl as single container without the redis overhead and run your backend containers autoscaled with an internal application load balancer. ECS will start a new resty container anyways when it's down (don't forget to setup health check properly).
And please challenge if this solution is really what you need. You have fully managed certificates for free from AWS. It's manageable up to some amount. Just use terraform to automate it.

rhlr commented

Yes, redis was the way forward. Also, by SSL offloading you meant that I should use TLS in front of the load balancer and only connect with port 80 of the container to generate certificates right ?

And please challenge if this solution is really what you need. You have fully managed certificates for free from AWS. It's manageable up to some amount. Just use terraform to automate it.

Can you please point me in the right direction (any links) from where I can acheive this ? Is this the one you're talking about ?

Public SSL/TLS certificates provisioned through AWS Certificate Manager are free. You pay only for the AWS resources you create to run your application.
https://aws.amazon.com/certificate-manager/pricing/

christian-s commented

SSL offloading = https<->http translation
SSL offloading in that context means, that you do your ssl offloading (this is basically what lua-resty-auto-ssl does + the auto ssl stuff) separate. Means NLB -> Target Groups[80,443] -> lua-resty-auto-ssl service -> Internal ALB -> Target Group[80] -> ECS services [containers http Port]

But as i pointed out, AWS has free certificates. This is absolutely recommended because it's fully managed and much more reliable than solutions like lua-resty-auto-ssl. lua-resty-auto-ssl adds much complexity and costs to your stack. You need good reasons like huge amount of domains to protect.

Steps:

  1. Optional and recommended: Add your domain to Route53 and configure the Route53 DNS servers in your domain
  2. Add your Domain to ACM (You can add multiple domains and also wildcards to your certificate)
  3. Verify your Domain using a DNS record as explained in ACM (or just press the add to rout53 button if you have that domain also there)
  4. Create an ALB
  5. Add an SSL Listener to the ALB using the ACM certificate
  6. Connect your ECS service target group (http/80)

Much easier and cheaper. Have fun!

  • 👍1