Certificate Authority failed to download the temporary challenge files created by Certbot.
Fresnoth opened this issue · 4 comments
I am having a similar issue to #195 as well as #54. However, after reading through both neither of those are exactly like this specific issue.
I thought that I might have a similar issue to #54 as I was trying to cert on a subdomain like -> xxxx.domain.com, however I didn't have a redirect in place as the subdomain was a listed A record. However, I went a step further took a different root domain that I owned and applied it to my server so I now have a single A record only on the domain pointing to my server plus the CNAME (www). I received the same error as before. I then looked at #195, however I am mapping port 80:80 and 443:443 in my docker compose file.
I am running the nginx container in front of my docker compose stack, with a slightly custom build, I am removing the redirector.conf however I am then using the same code within my own nginx.conf, using the jonasal/nginx-certbot:4.3.0 image.
This Dockerfile/nginx.conf works when using the "Local CA" method on my localhost however in production when I subbed in my own domain for the server_name localhost; lines it seems to not be allowing the connection from the cert authority.
I also verified my firewall settings, even going so far as to replace the nginx-certbot image with the normal nginx nginx:1.23-alpine image running just http and a slightly simpler nginx.conf and that allows a public connection to the stack on 80.
Docker Compose File:
version: '3.8'
services:
nginx:
build: ./compose/productionhttps/nginx
image: <image_name>
restart: unless-stopped
volumes:
- staticfiles:/app/staticfiles
- mediafiles:/app/mediafiles
- nginx_secrets:/etc/letsencrypt
env_file:
- ./.env/.prod-cert
ports:
- 80:80
- 443:443
- <port>:<port>
- <port>:<port>
depends_on:
- web
- <server2>
logging:
driver: syslog
options:
syslog-address: "<syslog_address>"
tag: "{{.Name}}/{{.ID}}"
env
#Docker-Nginix-Certbot - https://github.com/JonasAlfredsson/docker-nginx-certbot
# Required
CERTBOT_EMAIL='<email_address>'
# Optional (Defaults)
# DHPARAM_SIZE=2048
# ELLIPTIC_CURVE=secp256r1
# RENEWAL_INTERVAL=8d
# RSA_KEY_SIZE=2048
STAGING=1
# USE_ECDSA=1
# Advanced (Defaults)
# CERTBOT_AUTHENTICATOR=webroot
# CERTBOT_DNS_PROPAGATION_SECONDS=""
DEBUG=1
# USE_LOCAL_CA=1
# End Docker-Nginix-Certbot
Dockerfile
FROM jonasal/nginx-certbot:4.3.0
RUN rm /etc/nginx/conf.d/redirector.conf
COPY nginx.conf /etc/nginx/conf.d/
nginx.conf
server_tokens off;
ssl_protocols TLSv1.2 TLSv1.3;
upstream <server1> {
server <server1>:8000;
}
upstream <server2> {
server <server2>:<port>;
}
upstream <server3> {
server <server3>:<port>;
}
server {
# Listen on plain old HTTP and catch all requests so they can be redirected
# to HTTPS instead.
listen 80 default_server reuseport;
listen [::]:80 default_server reuseport;
# Anything requesting this particular URL should be served content from
# Certbot's folder so the HTTP-01 ACME challenges can be completed for the
# HTTPS certificates.
location '/.well-known/acme-challenge' {
default_type "text/plain";
root /var/www/letsencrypt;
}
# Everything else gets shunted over to HTTPS for each user defined
# server to handle.
location / {
return 301 https://$http_host$request_uri;
}
}
server {
# Listen to port 443 on both IPv4 and IPv6.
listen 443 ssl;
listen [::]:443 ssl;
server_name xxxxxx.com;
location / {
proxy_pass http://<server1>;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect off;
client_max_body_size 20M;
}
location /static/ {
alias /app/staticfiles/;
}
location /media/ {
alias /app/mediafiles/;
}
location /ws {
proxy_pass http://<server>;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
}
ssl_certificate /etc/letsencrypt/live/test-name/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/test-name/privkey.pem;
ssl_trusted_certificate /etc/letsencrypt/live/test-name/chain.pem;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}
server {
server_name xxxxxx.com;
listen <port>;
location / {
proxy_pass http://<server2>;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
}
server {
server_name xxxxxx.com;
listen <port>;
location / {
proxy_pass http://<server3>;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
}
Docker Logs
app-nginx-1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
app-nginx-1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
app-nginx-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
app-nginx-1 | 10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf is not a file or does not exist
app-nginx-1 | /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
app-nginx-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
app-nginx-1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
app-nginx-1 | /docker-entrypoint.sh: Configuration complete; ready for start up
app-nginx-1 | 2023/07/04 21:50:09 [debug] Debug messages are enabled
app-nginx-1 | 2023/07/04 21:50:09 [debug] Creating symlinks to any files found in /etc/nginx/user_conf.d/
app-nginx-1 | 2023/07/04 21:50:09 [warning] Could not find non-zero size keyfile file '/etc/letsencrypt/live/test-name/privkey.pem' in '/etc/nginx/conf.d/nginx.conf'
app-nginx-1 | 2023/07/04 21:50:09 [warning] Could not find non-zero size fullchain file '/etc/letsencrypt/live/test-name/fullchain.pem' in '/etc/nginx/conf.d/nginx.conf'
app-nginx-1 | 2023/07/04 21:50:09 [warning] Could not find non-zero size chain file '/etc/letsencrypt/live/test-name/chain.pem' in '/etc/nginx/conf.d/nginx.conf'
app-nginx-1 | 2023/07/04 21:50:09 [error] Important file(s) for '/etc/nginx/conf.d/nginx.conf' are missing or empty, disabling...
app-nginx-1 | 2023/07/04 21:50:09 [info] Starting the Nginx service in debug mode
app-nginx-1 | 2023/07/04 21:50:09 [debug] PID of the main Nginx process: 66
app-nginx-1 | 2023/07/04 21:50:09 [debug] RENEWAL_INTERVAL unset, using default of '8d'
app-nginx-1 | 2023/07/04 21:50:09 [info] Running the autorenewal service
app-nginx-1 | 2023/07/04 21:50:09 [debug] PID of the autorenewal loop: 69
app-nginx-1 | 2023/07/04 21:50:09 [debug] Creating symlinks to any files found in /etc/nginx/user_conf.d/
app-nginx-1 | 2023/07/04 21:50:09 [notice] 66#66: using the "epoll" event method
app-nginx-1 | 2023/07/04 21:50:09 [notice] 66#66: nginx/1.25.1
app-nginx-1 | 2023/07/04 21:50:09 [notice] 66#66: built by gcc 12.2.0 (Debian 12.2.0-14)
app-nginx-1 | 2023/07/04 21:50:09 [notice] 66#66: OS: Linux 5.10.0-23-cloud-amd64
app-nginx-1 | 2023/07/04 21:50:09 [notice] 66#66: getrlimit(RLIMIT_NOFILE): 1048576:1048576
app-nginx-1 | 2023/07/04 21:50:09 [notice] 66#66: start worker processes
app-nginx-1 | 2023/07/04 21:50:09 [notice] 66#66: start worker process 78
app-nginx-1 | 2023/07/04 21:50:09 [notice] 66#66: start worker process 87
app-nginx-1 | 2023/07/04 21:50:10 [info] Starting certificate renewal process
app-nginx-1 | 2023/07/04 21:50:10 [debug] Using staging environment
app-nginx-1 | 2023/07/04 21:50:10 [debug] RSA_KEY_SIZE unset, defaulting to 2048
app-nginx-1 | 2023/07/04 21:50:10 [debug] ELLIPTIC_CURVE unset, defaulting to 'secp256r1'
app-nginx-1 | 2023/07/04 21:50:10 [debug] Parsing config file '/etc/nginx/conf.d/nginx.conf.nokey'
app-nginx-1 | 2023/07/04 21:50:10 [debug] Found the following domain names: <domain_name><domain_name> <domain_name>
app-nginx-1 | 2023/07/04 21:50:10 [debug] Adding new key 'test-name' in array
app-nginx-1 | 2023/07/04 21:50:10 [info] Requesting an ECDSA certificate for 'test-name' (http-01 through webroot)
app-nginx-1 | Saving debug log to /var/log/letsencrypt/letsencrypt.log
app-nginx-1 | Requesting a certificate for <domain_name>
app-nginx-1 |
app-nginx-1 | Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
app-nginx-1 | Domain: <domain_name>
app-nginx-1 | Type: connection
app-nginx-1 | Detail: <ip_address>: Fetching http://<domain_name>/.well-known/acme-challenge/UrBrAXdzW4mB73PaEiIqEGYmVLH953RxRWrKtC_8cwI: Connection refused
app-nginx-1 |
app-nginx-1 | Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.
app-nginx-1 |
app-nginx-1 | Exiting abnormally:
app-nginx-1 | Traceback (most recent call last):
app-nginx-1 | File "/usr/local/bin/certbot", line 8, in <module>
app-nginx-1 | sys.exit(main())
app-nginx-1 | ^^^^^^
app-nginx-1 | File "/usr/local/lib/python3.11/dist-packages/certbot/main.py", line 19, in main
app-nginx-1 | return internal_main.main(cli_args)
app-nginx-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-nginx-1 | File "/usr/local/lib/python3.11/dist-packages/certbot/_internal/main.py", line 1864, in main
app-nginx-1 | return config.func(config, plugins)
app-nginx-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-nginx-1 | File "/usr/local/lib/python3.11/dist-packages/certbot/_internal/main.py", line 1597, in certonly
app-nginx-1 | lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
app-nginx-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-nginx-1 | File "/usr/local/lib/python3.11/dist-packages/certbot/_internal/main.py", line 141, in _get_and_save_cert
app-nginx-1 | lineage = le_client.obtain_and_enroll_certificate(domains, certname)
app-nginx-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-nginx-1 | File "/usr/local/lib/python3.11/dist-packages/certbot/_internal/client.py", line 517, in obtain_and_enroll_certificate
app-nginx-1 | cert, chain, key, _ = self.obtain_certificate(domains)
app-nginx-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-nginx-1 | File "/usr/local/lib/python3.11/dist-packages/certbot/_internal/client.py", line 428, in obtain_certificate
app-nginx-1 | orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
app-nginx-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-nginx-1 | File "/usr/local/lib/python3.11/dist-packages/certbot/_internal/client.py", line 496, in _get_order_and_authorizations
app-nginx-1 | authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
app-nginx-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-nginx-1 | File "/usr/local/lib/python3.11/dist-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
app-nginx-1 | self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
app-nginx-1 | File "/usr/local/lib/python3.11/dist-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
app-nginx-1 | raise errors.AuthorizationError('Some challenges have failed.')
app-nginx-1 | certbot.errors.AuthorizationError: Some challenges have failed.
app-nginx-1 | Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
Hi Fresnoth,
Your issue can be found in these lines:
app-nginx-1 | 2023/07/04 21:50:09 [warning] Could not find non-zero size keyfile file '/etc/letsencrypt/live/test-name/privkey.pem' in '/etc/nginx/conf.d/nginx.conf'
app-nginx-1 | 2023/07/04 21:50:09 [warning] Could not find non-zero size fullchain file '/etc/letsencrypt/live/test-name/fullchain.pem' in '/etc/nginx/conf.d/nginx.conf'
app-nginx-1 | 2023/07/04 21:50:09 [warning] Could not find non-zero size chain file '/etc/letsencrypt/live/test-name/chain.pem' in '/etc/nginx/conf.d/nginx.conf'
app-nginx-1 | 2023/07/04 21:50:09 [error] Important file(s) for '/etc/nginx/conf.d/nginx.conf' are missing or empty, disabling...
Nginx cannot start if there are configs that are incorrect or has missing certificate files, so the entrypoint script disables all configuration files that have not yet had their ceritficates downloaded. However, since you have the redirector settings (the port 80 stuff) in the same file we disable that one as well.
If you just let the redirector.conf file remain (and remove the duplicated code from your nginx.conf) you will probably see that it will start working.
Thank you so much Jonas!!
Feels bad that I missed that and didn't understand enough to get myself out but you are a true legend and super quick on the response!
I now just need to learn some more about nginx apparently. Thanks again you have a new sponsor.
Oh my, thank you so much for the donation, it is super appreciated! You made my day, and I will make sure this comes to great use :)
My pleasure helping, not everyone has time to spend as many hours battling Nginx quirks so don't feel bad about that, focus on that you now are a little bit more experienced ;)
I try to write down things I myself find weird so I will be able to remember in the future over here. if you feel something is missing or you want to add you are more than welcome to place a suggestion or do a pull request :)
P.S. You did a great work on providing a massive amount of details of your setup and logs, which made the debugging much easier. A maintainers dream when it comes to support issues ;)
No worries! Thanks again for your help and the link!