DietPi-Software | NGINX Proxy Manager (NPM)

[LilTrublMakr]

Sorry if this has been suggested before. I did search for npm and nginx proxy manager but did not see an issue for it.

Formal software information

• Software title | NGINX Proxy Manager (NPM)
• Short description | Reverse proxy manager, certificate manager
• Official URL (if available) | https://nginxproxymanager.com/
• GitHub URL (if available) | https://github.com/jc21/nginx-proxy-manager
• (Official) install documentation | https://nginxproxymanager.com/setup/

Are there similar/alternative software titles available with DietPi-Software?

NGINX (with a lot of manual setup)

What makes your requested software better than the above solutions, if available?

NPM provides a GUI that lets users make a simple reverse proxy, manage certificates, and much more. It is simple for newer people but allows you to get complicated as well.

How can DietPi make the installation easier or compatible, than following the install instructions or do APT installation, if available?

Though NPM is available for Docker only, it does allow the use of a SQLite database to keep everything self contained.

Can you provide the installation steps that you would suggest DietPi-Software to do?

1. Prerequisites: docker, docker-compose
2. Take the compose file from the installation documentation and modify it to use SQLite rather than a MySQL database.
3. Manually create the SQLite database file.*
4. Run docker-compose.

*One problem I did run in to is that I had to manually create the database.sqlite file by running a touch command because docker-compose was creating it as a directory rather than a file. It was starting the container but was throwing a Bad gateway error when trying to log in for the first time.

My example compose file:

version: "3"
services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: always
    ports:
      # Public HTTP Port:
      - '80:80'
      # Public HTTPS Port:
      - '443:443'
      # Admin Web Port:
      - '81:81'
      # Add any other Stream port you want to expose
      # - '21:21' # FTP
    environment:
      # If you would rather use Sqlite uncomment this
      # and remove all DB_MYSQL_* lines above
      DB_SQLITE_FILE: "/data/database.sqlite"
      # Uncomment this if IPv6 is not enabled on your host
      # DISABLE_IPV6: 'true'
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt

Are you willing to help maintaining the software installation, e.g. in case of needed setup changes due to updates etc.?

I can attempt to do the legwork of getting it added as an option, but support will be tough.

Vote for this software on FeatHub: https://feathub.com/MichaIng/DietPi/+101

[Joulinar]

Hi,

I don't think this will be implemented as it's a simple docker container that can be installed via docker-compose. Both docker and docker-compose are available via DietPi software catalogue

[LilTrublMakr]

I guess Portainer is more advanced to set up? It it not a "simple docker container" as well?

With NPM being one of the top 15 requests on FeatHub, I think it is something that should be considered as an exception to the 'no Docker containers' train of thought. Clearly, the want is there, and there is now a solution to keeping it self contained by using SQLite instead of having the hassle of goin through an external DB or starting a new DB with Docker. A lot of people struggle with reverse proxys and this is a really nice GUI solution that novices and pros alike can use.

[Joulinar]

Portainer is offered as management tool for Docker

For revers proxy, we are looking into frp at the moment #4395

[LilTrublMakr]

That looks very overly complicated to use. I have been looking at that for 5 mins and I have no clue what is going on besides looking like there are a ton of config files to edit. The GUI part looks to just be an information page with pretty graphs and numbers, nothing actually functional. Some people will like that, but I feel that it is not very novice friendly.

I hope more people will chime in and show interest for an official implementation. But for those that would like to run NPM without using an external database, here is how I got it to work:

1. Open a SSH terminal and navigate to where you keep your Docker container data. I create a subfolder for each of my containers so ./npm/ will be the file my docker-compose file will be in.
2. Create your docker-compose.yml file with the following content:

version: "3"
services:
  app:
    image: 'jc21/nginx-proxy-manager:2'
    restart: always
    ports:
      # Public HTTP Port:
      - '80:80'
      # Public HTTPS Port:
      - '443:443'
      # Admin Web Port:
      - '81:81'
      # Add any other Stream port you want to expose
      # - '21:21' # FTP
    environment:
      # If you would rather use Sqlite uncomment this
      # and remove all DB_MYSQL_* lines above
      DB_SQLITE_FILE: "/data/database.sqlite"
      # Uncomment this if IPv6 is not enabled on your host
      # DISABLE_IPV6: 'true'
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt

Modify as needed.

3. We need to create a database before starting the container since the docker container does not seem to create it. If we don't do this, you will get a Bad gateway error when logging in for the first time. Run this command: touch ./data/database.sqlite.
4. Start your container with the following command: docker-compose up -d.
5. When that is complete, you should be able to access NPM through your browser using http://dietpi:81 or <ip of DietPi>:81. The default log in credentials are UN [email protected] and PW changeme.

I hope someone finds this helpful.

[MichaIng]

SQLite indeed should be sufficient for the amount of data that a proxy usually holds. When it currently creates a missing DB_SQLITE_FILE as directory instead of as file, did you report this upstream? Good to have things fixed or otherwise clarified ASAP: https://github.com/jc21/nginx-proxy-manager/issues

I'm indeed no fan of implementing too simple software options, like "apt install package" or "pip install package" or other one-liner installs. Better to teach users how to use those package managers/installer tools and how to find available packages. In this case at least a config file needs to be manually created, a database chosen etc.

Let's see how difficult or easy we can make proxy setup via frp, which will be MUCH less overhead. Also some docs about how to setup a regular reverse proxy with a native HAProxy, Nginx, Apache2 or Lighttpd install would be good anyway. And if even with a proper documentation it is still too complicated for novice users, a GUI proxy tool turns out to be more relevant.

[Joulinar]

There seems to be a fix for the missing SQLite file on initial setup NginxProxyManager/nginx-proxy-manager#1122

---

EDIT

yep seems to be fixed.

As well it's working without docker compose as follow. All volumes on one place.

docker volume create npm_data
docker run -d -p 80:80 -p 443:443 -p 81:81 --name=npm --restart=always --env DB_SQLITE_FILE=/data/database.sqlite -v npm_data:/data -v npm_data:/etc/letsencrypt jc21/nginx-proxy-manager:github-pr-1122

quite straightforward

[Joulinar]

@MichaIng
If you agree, I could implement NPM for next release. It's quite a simple one

[naddel91]

@MichaIng If you agree, I could implement NPM for next release. It's quite a simple one

I agree! ;)

[Joulinar]

in meantime we decided against it as it's a simple command to have it running

[naddel91]

There seems to be a fix for the missing SQLite file on initial setup jc21/nginx-proxy-manager#1122

EDIT

yep seems to be fixed.

As well it's working without docker compose as follow. All volumes on one place.

docker volume create npm_data
docker run -d -p 80:80 -p 443:443 -p 81:81 --name=npm --restart=always --env DB_SQLITE_FILE=/data/database.sqlite -v npm_data:/data -v npm_data:/etc/letsencrypt jc21/nginx-proxy-manager:github-pr-1122

quite straightforward

Unable to find image 'jc21/nginx-proxy-manager:github-pr-1122' locally
docker: Error response from daemon: manifest for jc21/nginx-proxy-manager:github-pr-1122 not found: manifest unknown: manifest unknown.
See 'docker run --help'.

what am I missing?

[Joulinar]

jc21/nginx-proxy-manager:github-pr-1122 was a special image used for testing by NPM guys to validate an issue with SQLite. You would need to use the normal image in meantime

[naddel91]

jc21/nginx-proxy-manager:github-pr-1122 was a special image used for testing by NPM guys to validate an issue with SQLite. You would need to use the normal image in meantime

good to know. the normal image is just "jc21/nginx-proxy-manager"?

[Joulinar]

yes https://hub.docker.com/r/jc21/nginx-proxy-manager

[Joulinar]

the topic is bid older, but I like to share a small script that could be used to install/update NPM automatically. Close to what dietpi-software would do

#!/bin/bash
. /boot/dietpi/func/dietpi-globals

# Check for existing NginxProxyManager container
eval container=$(docker container ls -a | mawk '/jc21\/nginx-proxy-manager/{print $1;exit}')
[[ $container ]] && G_EXEC docker container rm -f "$container"
eval image=$(docker image ls -a | mawk '/jc21\/nginx-proxy-manager/{print $3;exit}')
[[ $image ]] && G_EXEC docker image rm "$image"
docker volume ls -q | grep -xq 'npm_data' || G_EXEC docker volume create npm_data

# Deploy the NginxProxyManager container
G_DIETPI-NOTIFY 2 'NginxProxyManager will be deployed now. This could take a while...'
G_EXEC_OUTPUT=1 G_EXEC docker run -d -p '80:80' -p '443:443' -p '81:81' --name=npm --restart=always --env 'DB_SQLITE_FILE=/data/database.sqlite' -v 'npm_data:/data' -v 'npm_data:/etc/letsencrypt' 'jc21/nginx-proxy-manager'

[holocronology]

I would tend to agree that given the value and functionality of Nginx Proxy Manager, it should be included as an installable package because it requires additional setting changes, such as the server port for lighttpd, to function properly.

[MichaIng]

What for do you need Lighttpd on the same machine?

[Joulinar]

It's web service for PiHole

[MichaIng]

Isn't it possible to serve it over the Nginx from NPM?

[Joulinar]

Don't think so. This would mean to modify Nginx server configuration within the container.

[MichaIng]

There is are no volumes for passing custom configs and webroot additions?

[holocronology]

Just to reiterate, if you have to muck around with additional configuration on the host device, this sort of negates the functionality and purpose of Docker.

On a side note, I had initially set up the NPM container on a macvlan network within docker. This gives the container it's own IP address on the host network, rather than having ports mapped via the host system. It appears that macvlan functionality within docker does not work properly on Dietpi. When I set the NPM container on macvlan, with an IP address of it's own on my network, it should have naturally seen port 80, 81, 443. It worked on the latter two, but the Dietpi host device continued to grab the port 80 traffic, even if sent to the IP address of the container and having set the port for lighttpd to 8080.

So, while @Joulinar's installation is seemingly working in their very basic configuration, it is not working for a number of other users--either in a base configuration or a more complex one that would eliminate any port issues.

[MichaIng]

this sort of negates the functionality and purpose of Docker.

If volumes exist for this, it is easier to create configs to serve web applications over NPM directly than installing a second webserver on the host first and create configs for it to serve web applications. Note the "if". If it is not foreseen, then I agree with you.

does not work properly on Dietpi

DietPi is Debian in all relevant regards. It does not "grab" any traffic, whatever you mean by this. Applications can listen on IP addresses or not and you can forward incoming traffic via iptables (and similar) or not, the exact same way it works on any other Debian based distribution 😉. If nothing is listening on/bound to port 80 on the host (check via ss -tlpn), and nothing arrives at your NPM at dedicated VLAN, then forwarding is not setup correctly. Also in theory it is possible that the kernel does not support needed nftables or VLAN features, e.g. the old Quartz64 kernel was very limited in this regards.

[Joulinar]

I guess it was/is an issue of the individual Docker configuration rather than a DietPi issue.

[holocronology]

If the DietPi (which I greatly approve of mind you) implementation of Docker does not support the full feature set of Docker networking (including macvlan and ipvlan) then this should be noted in the DietPi Software documentation.

Mar 06 21:10:03 HoloPi3B dockerd[494]: time="2023-03-06T21:10:03.700986549-05:00" level=warning msg="macvlan driver does not support port mappings"

*Yes, I'm likely coming across grouchy. I'm frustrated.

[Joulinar]

We install Docker-CE from official Docker apt repository. There is nothing special or DietPi specific. The only thing we do is to adjust data root and some log level / debug information.

DietPi/dietpi/dietpi-software

Lines 10350 to 10401 in b2395da

	if To_Install 162 # Docker
	then
	# Detect distro
	local distro='debian'
	(( $G_HW_MODEL < 10 )) && (( $G_RASPBIAN )) && distro='raspbian'
	# APT key
	local url="https://download.docker.com/linux/$distro/gpg"
	G_CHECK_URL "$url"
	G_EXEC eval "curl -sSfL '$url' | gpg --dearmor -o /etc/apt/trusted.gpg.d/dietpi-docker.gpg --yes"
	# APT list
	G_EXEC eval "echo 'deb https://download.docker.com/linux/$distro/ $G_DISTRO_NAME stable' > /etc/apt/sources.list.d/docker.list"
	G_AGUP
	# APT package
	# - Mask service to prevent iptables related startup failure: https://github.com/MichaIng/DietPi/issues/6013
	G_EXEC systemctl mask --now docker
	G_AGI docker-ce
	G_EXEC systemctl unmask docker
	G_EXEC systemctl start docker.socket
	# Change Docker service type to "simple": https://github.com/MichaIng/DietPi/issues/2238#issuecomment-439474766
	[[ -d '/lib/systemd/system/docker.service.d' ]] || G_EXEC mkdir /lib/systemd/system/docker.service.d
	G_EXEC eval "echo -e '[Service]\nType=simple' > /lib/systemd/system/docker.service.d/dietpi-simple.conf"
	# Config: https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file
	# - Move Docker containers to dietpi_userdata
	# - Log to systemd-journald (journalctl) by default with reduced log level: https://github.com/MichaIng/DietPi/issues/2388
	# + containerd: https://github.com/docker/docker.github.io/issues/9091
	[[ -d '/mnt/dietpi_userdata/docker-data' ]] || G_EXEC mkdir /mnt/dietpi_userdata/docker-data
	if [[ -f '/etc/docker/daemon.json' ]]
	then
	GCI_PRESERVE=1 G_CONFIG_INJECT '"data-root":' ' "data-root": "/mnt/dietpi_userdata/docker-data",' /etc/docker/daemon.json '^\{([[:space:]]|$)'
	GCI_PRESERVE=1 G_CONFIG_INJECT '"log-driver":' ' "log-driver": "journald",' /etc/docker/daemon.json '^\{([[:space:]]|$)'
	GCI_PRESERVE=1 G_CONFIG_INJECT '"log-level":' ' "log-level": "warn",' /etc/docker/daemon.json '^\{([[:space:]]|$)'
	GCI_PRESERVE=1 G_CONFIG_INJECT '"debug":' ' "debug": false,' /etc/docker/daemon.json '^\{([[:space:]]|$)'
	else
	[[ -d '/etc/docker' ]] || G_EXEC mkdir /etc/docker
	echo '{
	"data-root": "/mnt/dietpi_userdata/docker-data",
	"log-driver": "journald",
	"log-level": "warn",
	"debug": false
	}' > /etc/docker/daemon.json
	fi
	G_CONFIG_INJECT '\[debug\]' '[debug]' /etc/containerd/config.toml
	GCI_PRESERVE=1 G_CONFIG_INJECT 'level[[:blank:]]*=' ' level = "warn"' /etc/containerd/config.toml '^\[debug\]'
	Enable_memory_cgroup
	Configure_iptables
	fi

[holocronology]

I guess it was/is an issue of the individual Docker configuration rather than a DietPi issue.

This is the Docker Configuration. It doesn't get simpler than this.

version: "3"
services:
  app:
    image: jc21/nginx-proxy-manager:latest
    restart: unless-stopped
    ports:
      # These ports are in format <host-port>:<container-port>
      - 80:80 # Public HTTP Port
      - 443:443 # Public HTTPS Port
      - 81:81 # Admin Web Port
      # Add any other Stream port you want to expose
      # - '21:21' # FTP
      
    # Uncomment the next line if you uncomment anything in the section
    environment:
      # Uncomment this if you want to change the location of 
      # the SQLite DB file within the container
      # DB_SQLITE_FILE: "/data/database.sqlite"

      # Uncomment this if IPv6 is not enabled on your host
      DISABLE_IPV6: true
    
    volumes:
      - /mnt/dietpi_userdata/docker/npm/data:/data
      - /mnt/dietpi_userdata/docker/npm/letsencrypt:/etc/letsencrypt

It is virtually identical to your docker run script:

docker run -d -p '80:80' -p '443:443' -p '81:81' --name=npm --restart=always --env 'DB_SQLITE_FILE=/data/database.sqlite' -v 'npm_data:/data' -v 'npm_data:/etc/letsencrypt' 'jc21/nginx-proxy-manager'

[MichaIng]

If the DietPi (which I greatly approve of mind you) implementation of Docker does not support

I meant the kernel, not Docker itself, which may not support all features. However, also current Quartz64 images do fully support all this. Only the highly experimental RISC-V VisionFive 2 does not, but Docker doesn't start up there at all.

Did you check whether anything is listening on port 80 on the host? How did you setup the redirects? Are the VLANs up as expected (ip a)?

EDIT: Ah, let's not spam this software request. Feel free to open a new issue for investigating this case, instead.