Certbot fails to renew cerrtificates when they are already expired and force SSL is turned on

[valyrie97]

Checklist

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?
  • Yes
• Are you sure you're not using someone else's docker image?
  • Yes
• Have you searched for similar issues (both open and closed)?
  • Yes

Describe the bug

Renewing certificates fails when they are already expired and force SSL is turned on. when letsencrypt connects to nginx, it is forced to use an invalid certificate, and thus fails. the workaround is to disable forced SSL, renew, then re-enable.

Nginx Proxy Manager Version

2.9.18

To Reproduce
Steps to reproduce the behavior:

1. Setup a ___domain with letsencrypt and force SSL
2. wait for expiration to pass
3. try to renew
4. "Internal Error"

Expected behavior

I would expect it to not error, or at the very least tell me why the problem has occurred.

Screenshots

Operating System

Docker image running on archlinux kernel 5.18.5

Additional context

here is the logging that led me to the solution in /var/logs/letsencrypt/letsencrypt.log

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: [redacted]
  Type:   connection
  Detail: [redacted]: Fetching https://[redacted]/.well-known/acme-challenge/4KufgFpKv-p65j9aHzbJfmfSWneHYP-RdsxyYC_-QBk: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

2022-10-06 22:48:44,951:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2022-10-06 22:48:44,951:DEBUG:certbot._internal.error_handler:Calling registered functions
2022-10-06 22:48:44,951:INFO:certbot._internal.auth_handler:Cleaning up challenges
2022-10-06 22:48:44,951:DEBUG:certbot._internal.plugins.webroot:Removing /data/letsencrypt-acme-challenge/.well-known/acme-challenge/4KufgFpKv-p65j9aHzbJfmfSWneHYP-RdsxyYC_-QBk
2022-10-06 22:48:44,952:DEBUG:certbot._internal.plugins.webroot:All challenges cleaned up
2022-10-06 22:48:44,952:ERROR:certbot._internal.renewal:Failed to renew certificate npm-1 with error: Some challenges have failed.
2022-10-06 22:48:44,955:DEBUG:certbot._internal.renewal:Traceback was:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/renewal.py", line 484, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 1524, in renew_cert
    renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 127, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/renewal.py", line 344, in renew_cert
    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 441, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 493, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

[the1ts]

@valyrie97 you are correct PR #2038 has the fix and is awaiting merging and release. Either use the docker tag from that PR or turn off force SSL, manually renew and then turn on again.

[valyrie97]

@the1ts yes this worked great, thank you! Ive updated to the jc21/nginx-proxy-manager:github-pr-2038 image.