LetsEncrypt ACME redirect issue fixes #2881 #3121

EDIflyer · 2023-08-13T15:47:17Z

Evolution of #2038 to escape regex sequence (as per #2038 (comment)) and rebased against latest develop branch.

xipox · 2023-08-15T00:01:18Z

i don't know why NPM isn't starting when it's trying to migrate the DBs
`

app_1 | - /data/nginx/stream/17.conf
app_1 | - /data/nginx/stream/13.conf
app_1 | - /data/nginx/stream/7.conf
app_1 | - /data/nginx/stream/2.conf
app_1 | - /data/nginx/stream/3.conf
app_1 | ❯ Docker secrets ...
app_1 |
app_1 | -------------------------------------
app_1 | _ _ ____ __ __
app_1 | | \ | | _ | / |
app_1 | | | | |) | |/| |
app_1 | | |\ | __/| | | |
app_1 | || _|| || |_|
app_1 | -------------------------------------
app_1 | User: npm PUID:0 ID:0 GROUP:0
app_1 | Group: npm PGID:0 ID:0
app_1 | -------------------------------------
app_1 |
app_1 | ❯ Starting nginx ...
app_1 | ❯ Starting backend ...
app_1 | [8/14/2023] [11:52:05 PM] [Global ] › ℹ info Using Sqlite: /data/database.sqlite
app_1 | [8/14/2023] [11:52:06 PM] [Migrate ] › ℹ info Current database version: none
app_1 | [8/14/2023] [11:58:34 PM] [Global ] › ✖ error Command failed: . /opt/certbot/bin/activate && pip install --no-cache-dir certbot-dns-cloudflare==$(certbot --version | grep -Eo '0-9+') cloudflare && deactivate
app_1 | WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe76f73b240>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/certbot-dns-cloudflare/
app_1 | WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe76f73b518>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/certbot-dns-cloudflare/
app_1 | WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe76f73b6a0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/certbot-dns-cloudflare/
app_1 | WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe76f73b7f0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/certbot-dns-cloudflare/
app_1 | WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe76f73b940>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/certbot-dns-cloudflare/
app_1 | ERROR: Could not find a version that satisfies the requirement certbot-dns-cloudflare==2.5.0 (from versions: none)
app_1 | ERROR: No matching distribution found for certbot-dns-cloudflare==2.5.0
app_1 |
app_1 | [8/14/2023] [11:58:35 PM] [Migrate ] › ℹ info Current database version: none

`

EDIflyer · 2023-10-29T14:04:59Z

@jc21 any chance of merging this in? 🙂

JDENredden · 2023-11-21T05:56:08Z

This fixed my installation. Please merge.

EDIflyer · 2023-12-03T14:17:12Z

Thanks for approving @etabarestx 🙂 Do you know if a new release is upcoming with this and other PRs included?

KaeTuuN · 2023-12-11T20:25:37Z

Manually applied this fix via CLI and can confirm it works!

If anyone else doesn't want to wait for the merge, here is how to do it manually:

Open a shell as root in the docker container (varies, depending on your setup)
cd /etc/nginx/conf.d/include/ <- less typing
mv force-ssl.conf force-ssl.conf.bak <- don't change anything without a backup!
Copy the following codeblock:

cat <<'EOF' >force-ssl.conf     
# Since force-ssl.conf has now moved to the server section it overrides
# the LetsEncrypt config which is inside a ___location section
# Set FORCE variable in first two if tests pass and action in the third
set $FORCE "";
if ($scheme = "http") {
        set $FORCE 'H';
}
if ($request_uri !~ "^\/.well-known\/acme-challenge\/(.*)") {
        set $FORCE "${FORCE}D";
}
# If we are http and outside the LetsEncrypt directories redirect to https via 301
if ($FORCE = HD) {
        return 301 https://$host$request_uri;
}

Close your shell.
Restart the container.
REMINDER: This change will get lost if you update your Docker Image!

evansharp · 2024-03-15T17:30:25Z

Bumping to keep this alive.

smibrandon · 2024-04-01T18:19:33Z

I found a fix for my issue: allocating more storage space.

Running NPM in a Proxmox CT (no docker at all), and happened to catch that it was at 96% of its storage. I gave it some extra, and boom. Worked!

Blogshot · 2024-04-14T16:25:01Z

This has been an ongoing issue since 2022. How is this not merged yet?

As it stands, NPM will block renewals. I don't want to compromise security (-> disable "Force SSL") to enable renewals.

tristanXme · 2024-04-23T09:45:24Z

/bump please merge this!

Guiorgy · 2024-04-30T10:09:03Z

Applied this manually, but couldn't validate if it worked, since too many failed authorizations recently 😅.
Will update the comment later.
Edit: Didn't work for me, looks like I have a different problem.

HarryVasanth

This fixed my installation after manually applying this fix.
@jc21 could you please merge this when you have time? Thank you.

TailoredITRob · 2024-05-27T07:50:38Z

Manually applied this fix via CLI and can confirm it works!

If anyone else doesn't want to wait for the merge, here is how to do it manually:

Open a shell as root in the docker container (varies, depending on your setup)

cd /etc/nginx/conf.d/include/ <- less typing

mv force-ssl.conf force-ssl.conf.bak <- don't change anything without a backup!

Copy the following codeblock:
cat <<'EOF' >force-ssl.conf     
# Since force-ssl.conf has now moved to the server section it overrides
# the LetsEncrypt config which is inside a ___location section
# Set FORCE variable in first two if tests pass and action in the third
set $FORCE "";
if ($scheme = "http") {
        set $FORCE 'H';
}
if ($request_uri !~ "^\/.well-known\/acme-challenge\/(.*)") {
        set $FORCE "${FORCE}D";
}
# If we are http and outside the LetsEncrypt directories redirect to https via 301
if ($FORCE = HD) {
        return 301 https://$host$request_uri;
}
Close your shell.

Restart the container.

REMINDER: This change will get lost if you update your Docker Image!

Until this is committed, why not just create a force-ssl.conf file on your host machine. Then add it as a volume to the container. E.g...

force-ssl.conf (in the same directory as docker-compose.yml)

# Since force-ssl.conf has now moved to the server section it overrides
# the LetsEncrypt config which is inside a ___location section
# Set FORCE variable in first two if tests pass and action in the third
set $FORCE "";
if ($scheme = "http") {
        set $FORCE 'H';
}
if ($request_uri !~ "^\/.well-known\/acme-challenge\/(.*)") {
        set $FORCE "${FORCE}D";
}
# If we are http and outside the LetsEncrypt directories redirect to https via 301
if ($FORCE = HD) {
        return 301 https://$host$request_uri;
}

docker-compose.yml

services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      # These ports are in format <host-port>:<container-port>
      - '80:80' # Public HTTP Port
      - '443:443' # Public HTTPS Port
      - '81:81' # Admin Web Port
      # Add any other Stream port you want to expose
      # - '21:21' # FTP

    # Uncomment the next line if you uncomment anything in the section
    environment:
      # Uncomment this if you want to change the ___location of 
      # the SQLite DB file within the container
      # DB_SQLITE_FILE: "/data/database.sqlite"

      # Uncomment this if IPv6 is not enabled on your host
      DISABLE_IPV6: 'true'

    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
      - ./force-ssl.conf:/etc/nginx/conf.d/include/force-ssl.conf   # <~~~~~~~~~~~~~~~

This way if you update your container's base image, the force-ssl.conf change will persist.

lbogdan · 2024-05-31T08:27:26Z

docker/rootfs/etc/nginx/conf.d/include/force-ssl.conf

 }
+if ($request_uri !~ "^\/.well-known\/acme-challenge\/(.*)") {


The dot (.) should probably be escaped in the regex:

^\/\.well-known\/acme-challenge\/(.*)

Nice catch @lbogdan!
Yes, it would match not only .well-known/acme-challenge/ but also xwell-known/acme-challenge/, 1well-known/acme-challenge/, _well-known/acme-challenge/, etc.

@EDIflyer can you please push the change to the PR?

OK I've pushed that update although I see the main develop branch has changed from FORCE to TEST -

set $test ""; if ($scheme = "http") { set $test "H"; } if ($request_uri = /.well-known/acme-challenge/test-challenge) { set $test "${test}T"; } if ($test = H) { return 301 https://$host$request_uri; }

docker/rootfs/etc/nginx/conf.d/include/force-ssl.conf

It is therefore blocking this merge and won't autogenerate a container. Not sure if the underlying fix they have made resolved the issue or not?

The underlying fix has not resolved the issue. The URI can change per test, so your code works. We could refactor the existing to:

set $test ""; if ($scheme = "http") { set $test "H"; } if ($request_uri ~ "^\/\.well-known\/acme-challenge\/(.*)") { set $test "${test}T"; } if ($test = H) { return 301 https://$host$request_uri; }

Thanks @willyouvba - have updated the commit accordingly.

Sorry @EDIflyer, what I meant was: the contents of force-ssl.conf as it currently exists can be updated to the above (or modified completely to your version). Amalgamating the two won't work. It also looks like I lost a parenthesis at the end of the regex - I've updated my previous comment - sorry 😥.

Ahh, that explains why things went down for me overnight when Watchtower pulled the new image then 😂. Will try to change it to your revised version.

@willyouvba OK, can confirm sites back up and running using this new system!

Revised as per PR discussion.

EDIflyer · 2024-10-24T21:47:03Z

@jc21 @Nephiel I see some releases are happening again - any chance of merging this fix for the ACME redirect issue into the latest version so I can move off v2.11.2 onto it? (I saw there were some CVEs fixed)

Crazor · 2024-10-28T08:48:42Z

Can confirm that this is still an ongoing issue. I was really surprised that there is such an easy way to misconfigure NPM from the GUI such that SSL renewals just stop working. Please merge!

EDIflyer · 2024-11-07T16:41:59Z

@jc21 any chance of getting this merged in? Unfortunately the docker image of this PR seems to have been removed. Have created PR #4147 to update the rest of the code to the latest dev branch (and hope it generates a new docker image too!)

Re-do PR NginxProxyManager#3121 on latest development branch.

Guiorgy · 2024-11-07T17:50:46Z

@EDIflyer I'm assuming that fixing merge conflicts in the current PR would've generated a docker image, or at least that's what I'd hope. Have you tried doing that before creating a duplicate PR?

EDIflyer · 2024-11-07T21:25:36Z

@Guiorgy no I hadn't tried that as wasn't quite sure how to do so within Github. The new one built fine - will see if I can do anything to edit this current branch and whether it re-triggers anything.

EDIflyer · 2024-11-07T21:31:07Z

Hmm OK I've now click on my patch and updated the fork - seems to have triggered a build so if that works I'll close the other PR. Thanks for the suggestion - as it was 'stuck' I just didn't know how to get a new one built! Also I see it is now under a different ___location (nginxproxymanager/nginx-proxy-manager-dev:pr-XXXX rather than the old jc21/nginx-proxy-manager:github-pr-XXXX format which explains why Watchtower was giving errors!)

EDIflyer · 2024-11-07T23:48:52Z

@Guiorgy OK new docker container built OK so have closed the other PR - thanks for the suggestion :)

EDIflyer · 2024-12-31T08:09:20Z

@jc21 I see some other PRs have recently been merged in, any chance of accepting this one? I think it would close off quite a few issues.

PatVax · 2025-01-05T00:36:07Z

I had the same issue with renewing while "force ssl" is enabled. I switched to docker image nginxproxymanager/nginx-proxy-manager-dev:pr-3121 and I can confirm that this PR solves the issue at least in my case. I hope it will be merged soon.

mgkeeley · 2025-01-27T06:03:08Z

@jc21 Adding my voice to request that this PR is merged in.

EDIflyer · 2025-03-09T15:24:49Z

Updated PR to latest version of dev branch. Please remember to backup any databases before updating!

nginxproxymanagerci · 2025-03-09T15:53:31Z

Docker Image for build 8 is available on
DockerHub
as nginxproxymanager/nginx-proxy-manager-dev:pr-3121

Note: ensure you backup your NPM instance before testing this image! Especially if there are database changes
Note: this is a different docker image namespace than the official image

EDIflyer · 2025-03-09T21:43:17Z

Have updated from 2.12.2 to 2.12.3 - updated OK and able to renew cert OK with SSL still active.

HarryVasanth · 2025-03-14T10:14:51Z

Have updated from 2.12.2 to 2.12.3 - updated OK and able to renew cert OK with SSL still active.

Can confirm it's working as intended again. 👍

Crazor · 2025-04-10T07:37:56Z

I've updated from 2.12.3 to nginxproxymanager/nginx-proxy-manager-dev:pr-3121 and my certs renewed just fine, even with Force SSL enabled.
Can we get this merged?

LetsEncrypt ACME redirect issue fixes NginxProxyManager#2881

013310d

EDIflyer mentioned this pull request Aug 13, 2023

Temp workaround (that works for me!) for SSL certificate renewal bug #2881

Open

EDIflyer mentioned this pull request Aug 27, 2023

Internal Error trying to renew cert #396

Open

This was referenced Sep 4, 2023

SSL certificate error #2011

Open

SSL/Let's encrypt not working anymore after update or any other version #2713

Open

andzejsp mentioned this pull request Sep 13, 2023

JSON-Error in Backend while generating SSL-Certificates #2796

Open

This was referenced Oct 29, 2023

Internal Error when request new ssl certificate #3291

Open

Some challenges have failed. #3286

Open

Error when generating the SSL certificate (no Challenge!) #3250

Closed

etabarestx approved these changes Dec 3, 2023

View reviewed changes

KaeTuuN mentioned this pull request Dec 11, 2023

Update force-ssl.conf to allow for letsencrypt directories over http #2038

Open

Blogshot mentioned this pull request Apr 14, 2024

SSL cert not updating hassio-addons/addon-nginx-proxy-manager#400

Closed

HarryVasanth approved these changes May 7, 2024

View reviewed changes

lbogdan reviewed May 31, 2024

View reviewed changes

fix regex as per @lbogdan in NginxProxyManager#3121 (comment)

2e52834

HarryVasanth approved these changes May 31, 2024

View reviewed changes

EDIflyer added 2 commits June 10, 2024 20:10

Merge branch 'develop' into develop

2d65d67

Update force-ssl.conf

2bb99e6

Revised as per PR discussion.

TailoredITRob mentioned this pull request Jul 8, 2024

Cannot get new SSL certificates on Debian 12 with rootless Docker #3778

Closed

EDIflyer added a commit to EDIflyer/nginx-proxy-manager that referenced this pull request Nov 7, 2024

Update force-ssl.conf

a1408e2

Re-do PR NginxProxyManager#3121 on latest development branch.

EDIflyer mentioned this pull request Nov 7, 2024

Update force-ssl.conf #4147

Closed

Merge branch 'NginxProxyManager:develop' into develop

be5903f

EDIflyer mentioned this pull request Dec 5, 2024

Internal error on SSL certificates when force SSL is active #1625

Open

EDIflyer mentioned this pull request Jan 10, 2025

All certificate renewals suddenly fail. Has worked for months if not years. #2267

Open

Merge branch 'NginxProxyManager:develop' into develop

f5373dd

Merge branch 'NginxProxyManager:develop' into develop

9cac420

EDIflyer mentioned this pull request Mar 27, 2025

Error renewing and re-adding SSL Certificates #2233

Closed

		}
		if ($request_uri !~ "^\/.well-known\/acme-challenge\/(.*)") {

LetsEncrypt ACME redirect issue fixes #2881 #3121

Are you sure you want to change the base?

LetsEncrypt ACME redirect issue fixes #2881 #3121

Uh oh!

Conversation

EDIflyer commented Aug 13, 2023

Uh oh!

xipox commented Aug 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EDIflyer commented Oct 29, 2023

Uh oh!

JDENredden commented Nov 21, 2023

Uh oh!

EDIflyer commented Dec 3, 2023

Uh oh!

KaeTuuN commented Dec 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

evansharp commented Mar 15, 2024

Uh oh!

smibrandon commented Apr 1, 2024

Uh oh!

Blogshot commented Apr 14, 2024

Uh oh!

tristanXme commented Apr 23, 2024

Uh oh!

Guiorgy commented Apr 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HarryVasanth left a comment

Choose a reason for hiding this comment

Uh oh!

TailoredITRob commented May 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lbogdan May 31, 2024

Choose a reason for hiding this comment

Uh oh!

HarryVasanth May 31, 2024

Choose a reason for hiding this comment

Uh oh!

EDIflyer May 31, 2024

Choose a reason for hiding this comment

Uh oh!

willyouvba Jun 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

EDIflyer Jun 10, 2024

Choose a reason for hiding this comment

Uh oh!

willyouvba Jun 11, 2024

Choose a reason for hiding this comment

Uh oh!

EDIflyer Jun 11, 2024

Choose a reason for hiding this comment

Uh oh!

EDIflyer Jun 11, 2024

Choose a reason for hiding this comment

Uh oh!

EDIflyer commented Oct 24, 2024

Uh oh!

Crazor commented Oct 28, 2024

Uh oh!

EDIflyer commented Nov 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Guiorgy commented Nov 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EDIflyer commented Nov 7, 2024

Uh oh!

EDIflyer commented Nov 7, 2024

Uh oh!

EDIflyer commented Nov 7, 2024

Uh oh!

EDIflyer commented Dec 31, 2024

Uh oh!

PatVax commented Jan 5, 2025

xipox commented Aug 15, 2023 •

edited

Loading

KaeTuuN commented Dec 11, 2023 •

edited

Loading

Guiorgy commented Apr 30, 2024 •

edited

Loading

TailoredITRob commented May 27, 2024 •

edited

Loading

willyouvba Jun 10, 2024 •

edited

Loading

EDIflyer commented Nov 7, 2024 •

edited

Loading

Guiorgy commented Nov 7, 2024 •

edited

Loading

EDIflyer commented Mar 9, 2025 •

edited

Loading