Error renewing and re-adding SSL Certificates

[mandilpradhan]

Hello folks.

I'm experiencing a bizarre situation with the Let's Encrypt SSL Certificates on my NGINX Proxy Manager. It is installed on a Ubuntu VM (on Docker / Portainer using JC21 compose file) on my Proxmox server, and I am using DNS Challenge with a Cloudflare API to try to add the certificates. Ports 80 and 443 are forwarded and my services are reachable from outside my network.

1. The certificates did not auto-renew like it was supposed to when it expired. (screenshot 1)
2. I tried to manually renew them, but it failed with a simple "Internal Error" message. Nothing else. (screenshot 2)
3. When I deleted the certs and tried to re-add again, it gave me an error (screenshot 3) with the following error message:

Error: Command failed: certbot certonly --config "/etc/letsencrypt.ini" --cert-name "npm-6" --agree-tos --email "admin@mydomain.net" --domains "*.mydomain.net" --authenticator dns-cloudflare --dns-cloudflare-credentials "/etc/letsencrypt/credentials/credentials-6"
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Missing command line flag or config entry for this setting:
Please choose an account
Choices: ['c28983dcd32f@2022-06-07T04:34:33Z (781b)', 'c749392dabf7@2022-03-23T09:49:55Z (8a24)']
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

    at ChildProcess.exithandler (node:child_process:399:12)
    at ChildProcess.emit (node:events:526:28)
    at maybeClose (node:internal/child_process:1092:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)

Full logs for the error listed on # 3: https://pastebin.com/V5eQugEN

4. I removed the existing install of NGINX Proxy Manager and fired up a fresh install, and was able to add SSL certs for both MYDOMAIN.NET as well as *.MYDOMAIN.NET without any issues. (screenshot 4)
5. However, when I removed the SSL certs mentioned above and re-add them again to the same installation (screenshot 5), I got this error message:

Error: Command failed: /usr/sbin/nginx -t -g "error_log off;"
nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/npm-1/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/npm-1/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
nginx: configuration file /etc/nginx/nginx.conf test failed

    at ChildProcess.exithandler (node:child_process:399:12)
    at ChildProcess.emit (node:events:526:28)
    at maybeClose (node:internal/child_process:1092:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)

Full logs for the error listed on # 5 : https://pastebin.com/8Th8sCJn

Any help on this would be much appreciated!

Thanks!

[rallisf1]

I've been struggling with this for months. I host a couple dozen domains across 4 servers with NPM and only 1 updates the ssl certificates, for the rest 3 I have to recreate the hosts before the cert expires (which actually creates a new certificate, not renewing the existing).

[EDIflyer]

I'm having a similar issue. I've got one server where NPM seems to be working and auto-renewing OK but another where I get errors about auto-renewal (never used to have such issues) and for a couple of the hosts I managed to delete the SSL cert and recreate it but now I'm running into that BIO_new_file() failed error too.

[EDIflyer]

Still having issues with this on one site. Sometimes deleting the existing SSL certificate and recreating from scratch works.

2022-12-14 00:03:17,027:DEBUG:certbot._internal.display.obj:Notifying user: 
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: [redacted]
  Type:   connection
  Detail: [redacted]: Fetching https://[redacted]/.well-known/acme-challenge/SrwlbiCpyhLQIBurWFCVqLH9-jwzpPXkQPEX0mQ7VA4: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

2022-12-14 00:03:17,029:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2022-12-14 00:03:17,029:DEBUG:certbot._internal.error_handler:Calling registered functions
2022-12-14 00:03:17,029:INFO:certbot._internal.auth_handler:Cleaning up challenges
2022-12-14 00:03:17,029:DEBUG:certbot._internal.plugins.webroot:Removing /data/letsencrypt-acme-challenge/.well-known/acme-challenge/SrwlbiCpyhLQIBurWFCVqLH9-jwzpPXkQPEX0mQ7VA4
2022-12-14 00:03:17,029:DEBUG:certbot._internal.plugins.webroot:All challenges cleaned up
2022-12-14 00:03:17,030:ERROR:certbot._internal.renewal:Failed to renew certificate npm-6 with error: Some challenges have failed.
2022-12-14 00:03:17,032:DEBUG:certbot._internal.renewal:Traceback was:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/renewal.py", line 484, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 1541, in renew_cert
    renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 129, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/renewal.py", line 344, in renew_cert
    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 442, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 510, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2022-12-14 00:03:17,032:DEBUG:certbot._internal.display.obj:Notifying user: 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2022-12-14 00:03:17,032:ERROR:certbot._internal.renewal:All renewals failed. The following certificates could not be renewed:
2022-12-14 00:03:17,033:ERROR:certbot._internal.renewal:  /etc/letsencrypt/live/npm-6/fullchain.pem (failure)
2022-12-14 00:03:17,033:DEBUG:certbot._internal.display.obj:Notifying user: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2022-12-14 00:03:17,033:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/opt/certbot/lib/python3.7/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 1744, in main
    return config.func(config, plugins)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 1630, in renew
    renewal.handle_renewal_request(config)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/renewal.py", line 511, in handle_renewal_request
    f"{len(renew_failures)} renew failure(s), {len(parse_failures)} parse failure(s)")
certbot.errors.Error: 1 renew failure(s), 0 parse failure(s)
2022-12-14 00:03:17,034:ERROR:certbot._internal.log:1 renew failure(s), 0 parse failure(s)

and another

2022-12-14 00:07:07,326:INFO:certbot._internal.auth_handler:Performing the following challenges:
2022-12-14 00:07:07,326:INFO:certbot._internal.auth_handler:http-01 challenge for [redacted]
2022-12-14 00:07:07,326:INFO:certbot._internal.plugins.webroot:Using the webroot path /data/letsencrypt-acme-challenge for all unmatched domains.
2022-12-14 00:07:07,327:DEBUG:certbot._internal.plugins.webroot:Creating root challenges validation dir at /data/letsencrypt-acme-challenge/.well-known/acme-challenge
2022-12-14 00:07:07,337:DEBUG:certbot._internal.plugins.webroot:Attempting to save validation to /data/letsencrypt-acme-challenge/.well-known/acme-challenge/NyTX9jEpd9OjcKAhiSMbmvMhnUGybRsQeodbiwKu5z8
2022-12-14 00:07:07,344:DEBUG:acme.client:JWS payload:
b'{}'
2022-12-14 00:07:07,346:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/chall-v3/186808196477/B_qo8A:
{
  "protected": "[redacted]",
  "signature": "[redacted]",
  "payload": "e30"
}
2022-12-14 00:07:07,474:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/chall-v3/186808196477/B_qo8A HTTP/1.1" 404 102
2022-12-14 00:07:07,475:DEBUG:acme.client:Received response:
HTTP 404
Server: nginx
Date: Wed, 14 Dec 2022 00:07:07 GMT
Content-Type: application/problem+json
Content-Length: 102
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 1AAD6IebNbKrK1FjKWXEdyKxN6OAAmQcBCe9xfwofU8Hvfc

{
  "type": "urn:ietf:params:acme:error:malformed",
  "detail": "No such challenge",
  "status": 404
}
2022-12-14 00:07:07,476:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 102, in handle_authorizations
    self.acme.answer_challenge(achall.challb, resp)
  File "/opt/certbot/lib/python3.7/site-packages/acme/client.py", line 192, in answer_challenge
    resp = self._post(challb.uri, response)
  File "/opt/certbot/lib/python3.7/site-packages/acme/client.py", line 114, in _post
    return self.net.post(*args, **kwargs)
  File "/opt/certbot/lib/python3.7/site-packages/acme/client.py", line 1289, in post
    return self._post_once(*args, **kwargs)
  File "/opt/certbot/lib/python3.7/site-packages/acme/client.py", line 1303, in _post_once
    response = self._check_response(response, content_type=content_type)
  File "/opt/certbot/lib/python3.7/site-packages/acme/client.py", line 1149, in _check_response
    raise messages.Error.from_json(jobj)
acme.messages.Error: urn:ietf:params:acme:error:malformed :: The request message was malformed :: No such challenge

2022-12-14 00:07:07,476:DEBUG:certbot._internal.error_handler:Calling registered functions
2022-12-14 00:07:07,476:INFO:certbot._internal.auth_handler:Cleaning up challenges
2022-12-14 00:07:07,476:DEBUG:certbot._internal.plugins.webroot:Removing /data/letsencrypt-acme-challenge/.well-known/acme-challenge/NyTX9jEpd9OjcKAhiSMbmvMhnUGybRsQeodbiwKu5z8
2022-12-14 00:07:07,476:DEBUG:certbot._internal.plugins.webroot:All challenges cleaned up
2022-12-14 00:07:07,477:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/opt/certbot/lib/python3.7/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 1744, in main
    return config.func(config, plugins)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 1591, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 141, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 530, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 442, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 510, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 102, in handle_authorizations
    self.acme.answer_challenge(achall.challb, resp)
  File "/opt/certbot/lib/python3.7/site-packages/acme/client.py", line 192, in answer_challenge
    resp = self._post(challb.uri, response)
  File "/opt/certbot/lib/python3.7/site-packages/acme/client.py", line 114, in _post
    return self.net.post(*args, **kwargs)
  File "/opt/certbot/lib/python3.7/site-packages/acme/client.py", line 1289, in post
    return self._post_once(*args, **kwargs)
  File "/opt/certbot/lib/python3.7/site-packages/acme/client.py", line 1303, in _post_once
    response = self._check_response(response, content_type=content_type)
  File "/opt/certbot/lib/python3.7/site-packages/acme/client.py", line 1149, in _check_response
    raise messages.Error.from_json(jobj)
acme.messages.Error: urn:ietf:params:acme:error:malformed :: The request message was malformed :: No such challenge
2022-12-14 00:07:07,478:ERROR:certbot._internal.log:An unexpected error occurred:
2022-12-14 00:07:07,482:ERROR:certbot._internal.log:No such challenge

[Tharic99]

Out of curiosity, are any of you using an Access List on the host that you are unable to renew your SSL cert against?

I was having this error on a few SSL certs myself and just realized that all of them are using an Access List that requires authentication. When I temporarily REMOVED the Access List from the host(s) using the SSL certs, and set them to simply "Public", I was able to then renew the SSL cert without any problems at all.

Then once it was renewed, I put the Access List with authentication back on.

[EDIflyer]

@Tharic99 no access list in use here. I can't quite work it out as for some a plain renew SSL didn't work but generating a new certificate did. However the old ones don't seem to be deleted cleanly by NPM when you do so via the SSL page (at least not always!), sometimes getting stuck and requiring manual intervention.

[EDIflyer]

Just to update, this has happened again and clearing certbot instances doesn't help, so I'm having to delete each SSL cert and then request a new one on a one by one basis.

[github-actions]

Issue is now considered stale. If you want to keep it open, please comment 👍

[github-actions]

Issue was closed due to inactivity.

[EDIflyer]

FYI PR #3121 resolves this issue