Cannot get new SSL certificates on Debian 12 with rootless Docker

[TailoredITRob]

Checklist

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?
  • Yes
• Are you sure you're not using someone else's docker image?
  • Yes
• Have you searched for similar issues (both open and closed)?
  • Yes (Possible duplicates listed)

Describe the bug
I have a vanilla installation of Debian 12 with rootless Docker and NPM installed. (I also have a container running Portainer.) When I try to enable SSL and add a new certificate, it fails. When I attempt to create a new Let'sEncrypt certificate from the SSL page, it also fails. Additionally, the Server Reachability test also fails with the following error.

There is a server found at this ___domain but it returned an unexpected status code 502. Is it the NPM server? Please make sure your ___domain points to the IP where your NPM instance is running.

Of course, NPM is running and I'm accessing NPM through the ___domain I'm attempting to pull an SSL cert for.

Inspecting the JS console, I find a request going out to http://my.___domain.com/api/nginx/certificates/test-http?domains=["my.___domain.com"] is failing with the following error.

{
    "error":
        {
            "code":403,
            "message": "Permission Denied"
        },
    "debug":
        {
            "stack":
                [
                    "PermissionError: Permission Denied",
                    "    at /app/lib/access.js:309:13",
                    "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
                    "    at async Object.testHttpsChallenge (/app/internal/certificate.js:1131:3)"
                ],
            "previous":
                {
                    "name": "PermissionError",
                    "message": "Permission Denied",
                    "public": true,
                    "status": 403,
                    "permission": "certificates:list"
                }
        }
}

Attempting to request a new certificate results in a 503 error from http://my.___domain.com/api/nginx/certificates with the following message.

CommandError: Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

The indicated log file is never created.

Nginx Proxy Manager Version
2.11.2

To Reproduce
Steps to reproduce the behavior:

1. Perform a clean install of Debian 12.
2. Install docker-ce (rootless).
3. Use the docker-compose.yml file shown below.
4. Log into NPM with default credentials, create a new password.
5. Add a new host for the any ___domain and attempt to add a new SSL certificate or run a reachability test.

Expected behavior
A reachability test should be completed or a new Let'sEncrypt certificate issued without errors.

Operating System
Debian GNU/Linux 12 (bookworm)

Additional context

services:
  portainer:
    image: 'portainer/portainer-ce:latest'
    restart: unless-stopped
    ports:
      - '127.0.0.1:9443:9443'
    volumes:
      - './storage/data:/data'
      - '/etc/localtime:/etc/localtime:ro'
      # - '/var/run/docker.sock:/var/run/docker.sock'
      - '$XDG_RUNTIME_DIR/docker.sock:/var/run/docker.sock'

  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      - '80:80' # Public HTTP Port
      - '443:443' # Public HTTPS Port
      - '81:81' # Admin Web Port
    environment:
      # Mysql/Maria connection parameters:
      DB_MYSQL_HOST: "db"
      DB_MYSQL_PORT: 3306
      DB_MYSQL_USER: "npm"
      DB_MYSQL_PASSWORD: "password"
      DB_MYSQL_NAME: "npm"
      # Uncomment this if IPv6 is not enabled on your host
      DISABLE_IPV6: 'true'
    volumes:
      - './storage/data:/data'
      - './storage/letsencrypt:/etc/letsencrypt'
    depends_on:
      - db

  db:
    image: 'jc21/mariadb-aria:latest'
    restart: unless-stopped
    environment:
      MYSQL_ROOT_PASSWORD: 'rootpassword'
      MYSQL_DATABASE: 'npm'
      MYSQL_USER: 'npm'
      MYSQL_PASSWORD: 'password'
      MARIADB_AUTO_UPGRADE: '1'
    volumes:
      - './storage/mysql:/var/lib/mysql'

Possibly related to #2439, #2593, and #2713.

[TailoredITRob]

I finally have this resolved.

I needed to apply the fix from #3121 as shown below.

Create a force-ssl.conf file in the same directory as docker-compose.yml and then add it to your container.

force-ssl.conf

set $test "";
if ($scheme = "http") {
    set $test "H";
}
if ($request_uri ~ "^\/\.well-known\/acme-challenge\/(.*)") {
    set $test "${test}T";
}
if ($test = H) {
    return 301 https://$host$request_uri;
}

docker-compose.yml

services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      # These ports are in format <host-port>:<container-port>
      - '80:80' # Public HTTP Port
      - '443:443' # Public HTTPS Port
      - '81:81' # Admin Web Port
      # Add any other Stream port you want to expose
      # - '21:21' # FTP

    # Uncomment the next line if you uncomment anything in the section
    environment:
      # Uncomment this if you want to change the ___location of 
      # the SQLite DB file within the container
      # DB_SQLITE_FILE: "/data/database.sqlite"

      # Uncomment this if IPv6 is not enabled on your host
      DISABLE_IPV6: 'true'

    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
      - ./force-ssl.conf:/etc/nginx/conf.d/include/force-ssl.conf   # <~~~~~~~~~~~~~~~

This should resolve the issue until PR #3121 is merged. With this I was getting back Let's Encrypt responses, but they were being denied. After inspecting the logs I realized it was failing a CAA check. (I've used other certificate providers for other subdomains. CAA records limit which CAs may issue certificates for a ___domain. You may need to add a CAA record for Let's Encrypt.

example.org         CAA 0 issue "letsencrypt.org"         3600

You can learn more about CAA records here.