Automated docker build for nomadnet daemon

[jphastings]

Uses a Github Action (.github/workflows/publish-container.yml) to create a docker container image (from Dockerfile) that represents an extremely minimal python installation with a virtualenv holding all requirements necessary to
execute nomadnet.

Once nomadnet supports a daemon mode, we can have those commandline options/config file defaults be present within the default image too (though now it has no special default config, so it only works interactively; see the first docker run example below.)

New docker images are created on pushes to master or pushes to tags matching *.*.* (ie. version tags) and are retrievable with those tags. It's a "multistage build", ie. the heavy lifting (of compiling the rust) is done in a throw-away container, nomadnet is installed to a virtualenv, which is the only thing copied over to the published container, to reduce its size (from ~989MB to ~93MB).

The Github action is free for public repos (docs); you can see the output from one of my test runs here (expand the "Build and puhs docker image" row for the build output).

Other notable changes:

• I've added a short piece in the README.md on how to use Docker (wording amendments welcome!)
• I've added a .dockerfileignore (which is a symlink to .gitignore) — this ensures no files that wouldn't be in the git repo are accidentally used as part of the build process. I don't think we need to manage the ignore list separately for now, though there are slight differences in their file formats, if your .gitignore needs to get complex.
• The action in .github/workflow/ is straight out of the github readme. There might be some additions we want to make, but it works great as-is!

Examples (will run once merged; replace markqvist with jphastings below to test now):

$ docker pull ghcr.io/markqvist/nomadnet:master

# Print docker labels, to demonstrate the image has been retrieved
$ docker inspect -f '{{json .Config.Labels}}' ghcr.io/markqvist/nomadnet:master | jq
{
  "org.opencontainers.image.created": "2022-04-27T06:01:55.894Z",
  "org.opencontainers.image.description": "Communicate Freely",
  "org.opencontainers.image.licenses": "GPL-3.0",
  "org.opencontainers.image.revision": "59cffc4a9de0f276d2cc87537ff1316aed5f16dd",
  "org.opencontainers.image.source": "https://github.com/markqvist/NomadNet",
  "org.opencontainers.image.title": "NomadNet",
  "org.opencontainers.image.url": "https://github.com/markqvist/NomadNet",
  "org.opencontainers.image.version": "master"
}

# Run nomadnet interactively without installing it (with default config)
$ docker run -it ghcr.io/markqvist/nomadnet:master

# Run nomadnet as a daemon, using config stored on the host machine in specific directories
$ docker run -d -v /local/path/nomadnetconfig/:/root/.nomadnetwork/ -v /local/path/reticulumconfig/:/root/.reticulum/:rw ghcr.io/markqvist/nomadnet:master

[markqvist]

This is beautiful. Thank you so much for taking the initiative and setting this up.

Sorry for my ignorance here, I am at a really basic level when it comes to docker, but how does docker handle networking? Is it completely isolated from the host, or is "localhost" in the running docker image the same as "localhost" on the docker host system?

I am asking because we should probably consider how the nomadnet daemon in the image connects to any already running RNS daemon on the host, or if it should run it's own rnsd internally (and possibly have access to USB ports or networking devices of the host). If what I am saying doesn't even make sense, please correct me ;)

[jphastings]

My pleasure! It's always nice to share the value from my day-job skills!

So docker has a variety of networking options that the end-user can take advantage of without us having to do much to prep for them. By default it creates a bridge network for containers like this being run as an image (by default all containers are on the same bridge network). Outbound traffic works as expected without changes, but inbound isn't hooked up unless the ports are exposed at run time (they can be "EXPOSE"d in the Dockerfile for discoverability too). Users can also docker run --network host <etc> and the container will run as if it were a new machine on your network, but folks who know to do that already know what they're doing enough to have no trouble there.

With bridge networking, if NomadNet makes only outbound connections then we're already set (until we dockerize rnsd and want to share data between images 😉), but if we want to also allow inbound network connections (eg. for the mesh networking) then we need to:

• ensure the port being opened by NomadNet is predictable, so we can add it to the Dockerfile
• add an EXPOSE <port> declaration to the Dockerfile (optionally with a tcp/udp protocol declaration, if we care to be specific)
• instruct our users to use the -p or -P flags appropriately (see below)
• recognise/adjust our code as NomadNet won't be receiving traffic on the same port number that it knows it has open (it's basically a NAT; so nomadnet's application code won't know the IP address or port it can be contacted on externally, as it'll be different)

We don't need to take care of all possibilities here, just the default ones. Users of this image will run docker run -P [etc]/nomadnet to have nomadnet's inbound port(s) mapped to (randomly assigned) host port(s) (or can use an explicit declaration instead -p <host port>:<NomadNet port> if they want precision) if they want a port exposed.

An aside; there's a slight tension here with the way NomadNet is configured — via a config file — and Docker's conventional 12 factor app approach of using environment variables for config. The approach I've taken with this PR is to allow users of this image to run their container mounting local folder(s) with config, and also expecting them to add the appropriate flags to their docker run command to allow the features their config file requests. (Particularly access to USB devices, you'd need to docker run --privileged to get total access, or docker run --device=/dev/whatever:/dev/whatever <etc> to share only a single device — as per the docs)

If we wanted to make things "more docker" we could have the command the docker image runs be a script that compiles environment variables into a config file, then runs nomadnet; but then you have an extra dependency to manage when you change the structure of the config file and so on. Combine that with that fact that most folks using docker (instead of pip to run nomadnet) will probably keep their config the same for the vast majority of the time, I figure what we have here is plenty enough for most people (unless you have other desires to use env vars for config).

Back to your question for the last part; we should not assume that the app running in the docker container can to talk to the host at all (the only case where that's true is when --network host is used, and even then you'd need to know the IP address of the host, as the container runs as if it were a different device on the network). Instead we would either:

• Do what I've done here, and use the "in-built" RNS to run nomadnet, so no external network connection is needed
• Also have rnsd available as a docker image, which can be run separately and would then be accessible from this nomadnet one. This is because, by default, all docker containers run on the same bridge network with /etc/hosts populated so that host lookups reflect what's added by the user in the --name flag to docker run for whatever containers they have running. This means we can advise people to docker run --name my_rnsd rnsd then have the address of the rnsd for nomadnet be my_rnsd instead of localhost. (There's a whole aside into docker-compose here, which is a language & tool for specifying & running collections of docker images in unison that seems like it'd be useful here)

Again, this feels like moderately advanced stuff that people who use docker regularly will already be familiar enough with to be able to make progress without our explicit efforts; but if there are any particular use cases you want to make super easy, we can optimise all of this for them without much hassle.

I hope this all makes sense; I'm very happy to keep chatting on GitHub here async, but I'm also on-call (and so stuck reasonably close to a computer) this weekend, so if you wanted to have a video chat to talk through the implications of any of this I'll have time (and inclination!) somewhere in this Sat/Sun/Mon for that (I'm based in London, so I'm awake/about broadly BST working hours).

Apologies for the long message 😅 hope you're enjoying the day!

[markqvist]

Ok, thanks so much for the detailed description! If you don't mind, it'd be awesome if you have time for a voice/video call on the subject, there's a few things I'm still not completely clear about ;) Are you on the matrix network? I think my questions are primarily related to understanding what level of exposure to the internals of an image, that the average docker user expects, and how to make it as easy as possible to set it up for people.

I'll merge this now, just to get it in there, we can always adjust the details if needed.

[jphastings]

@markqvist I am on matrix now, though I can't claim I know what I'm doing yet 😅 I'll be happy to take a call after my work hours today (say, after 17:30 BST?); I've started a conversation with you (using the email address listed by github), let me know if that works!

[markqvist]

Thanks! I've been on the road for a few days with intermittent connectivity, but I joined the room now, will try and catch you there.