Initial draft #2

TDDR · 2022-02-27T17:40:32Z

This is my initial draft of how I think the docker-compose, registry-config, and the nginx.config should look like.

I know there is still a lot of work to be done but I figured I would get something up so we can start collaborating.

I included the links for the templates I used to make each of the files at the top of each file. I made altercations to both the docker-compose and the registry-config. The nginx.conf however was a straight cut and paste from the linked website.

humphd

Thanks for getting this started. I've left some initial thoughts.

config/nginx.conf

humphd · 2022-02-27T18:25:00Z

config/nginx.conf

+
+    # SSL
+    ssl_certificate /etc/nginx/conf.d/domain.crt;
+    ssl_certificate_key /etc/nginx/conf.d/domain.key;


@manekenpix has certs, he'll know what he wants done here.

humphd · 2022-02-27T18:25:58Z

config/nginx.conf

+
+      # To add basic authentication to v2 use auth_basic setting.
+      auth_basic "Registry realm";
+      auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;


This is probably fine to start, until we put in a better GitHub based auth solution.

We'll need to generate nginx.htpasswd with some users to test.

config/registry-config.yaml

humphd · 2022-02-27T18:27:24Z

config/registry-config.yaml

+    service: registry
+storage:
+  cache:
+    blobdescriptor: inmemory # or redis?


Memory is fine for now.

humphd · 2022-02-27T18:28:37Z

config/registry-config.yaml

+http:
+  addr: 443:5000 # Why the 443 -> https://docs.docker.com/registry/configuration/#letsencrypt
+  host: docker.cdot.systems
+  secret: sUperDuperSecret


We can't do this. We'll need an option for how we add this (env?)

docker-compose.yaml

humphd · 2022-02-27T18:29:59Z

docker-compose.yaml

+      REGISTRY_AUTH_TOKEN_ISSUER: Registry token issuer
+      REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE: /root/certs/bundle
+    volumes:
+      - /telescope/data:/var/lib/registry


We should figure out where in the host we're going to put our stuff.

I think /usr/share/our_directory would make sense.

When you say our_directory, would that mean this directory, /mnt/docker0storage/registry?

Correct, so it would be:

volumes: - /mnt/docker0storage/registry:/var/lib/registry

Such that the container will overlay /mnt/docker0storage/registry onto /var/lib/registry within the container.

docker-compose.yaml

humphd · 2022-02-28T00:54:05Z

docker-compose.yaml

+#https://docs.docker.com/registry/deploying/#deploy-your-registry-using-a-compose-file
+services:
+  nginx:
+    image: "nginx:alpine"


Let's use nginx:stable-alpine. Boring and safe vs cutting edge!

Do we need to use bcrypt?

nginx:stable-alpine is still alpine, just a stable vs. development version of nginx

docker-compose.yaml

humphd · 2022-02-28T00:57:27Z

docker-compose.yaml

+    ports:
+      - 5043:443
+    restart: unless-stopped
+    links:


I don't think we need this, since our registry is named registry already, and will be reachable by nginx.

humphd · 2022-02-28T00:58:10Z

docker-compose.yaml

+    links:
+      - registry:registry
+    volumes:
+      - ./auth:/etc/nginx/conf.d


@manekenpix where should we put our files in the host? It might make sense to have ./nginx but I don't think auth makes as much sense for these.

I think a config folder would be ok?

I had thought the same thing originally, but after going back and looking at the documentation I saw this.

from here https://docs.docker.com/registry/recipes/nginx/#setting-things-up

So I decided to go with the current folder structure. I can create and move it back to a config folder though?

I think the easiest way to do this is to mimic the layout of what we'll pass in within our repo:

docker-compose.yml config/ (or this could be called volume/) etc/ nginx/ conf.d

This way you can look at the overlays we have in our repo, and understand quickly where they end-up in the container.

volumes: - ./config/etc/nginx/conf.d:/etc/nginx/conf.d

humphd · 2022-02-28T00:58:26Z

docker-compose.yaml

+    image: certbot/certbot
+    container_name: 'certbot'
+    volumes:
+      - ../../certbot/conf:/etc/letsencrypt


Similar question here, we need to determine our local path

Maybe somewhere in /etc, like /etc/letsencrypt?

Yeah, config/etc/letsencrypt would work similar to my comment above.

docker-compose.yaml

manekenpix · 2022-02-28T01:21:50Z

auth/nginx.conf

+
+  server {
+    listen 443 ssl;
+    server_name docker.cdot.systems;


I think we should use a variable here, in case we decide to move it somewhere else.

Suggested change

server_name docker.cdot.systems;

server_name ${DOMAIN};

Kevan-Y · 2022-02-28T01:29:31Z

auth/nginx.conf

+    # required to avoid HTTP 411: see Issue #1486 (https://github.com/moby/moby/issues/1486)
+    chunked_transfer_encoding on;
+
+    location /v2/ {


Just to make sure I understand. We will push our image to DOMAIN/v2 route?

Kevan-Y · 2022-02-28T01:37:27Z

registry-config.yaml

+    rootdirectory: /mnt/docker0storage/registry
+http:
+  addr: 443:5000 # https://docs.docker.com/registry/configuration/#letsencrypt
+  host: docker.cdot.systems


Could we change it to ${DOMAIN} like what @manekenpix suggested in one of the other file?

If we do, we'll have to use envsubst or something to replace the values before we use this file.

DukeManh · 2022-02-28T04:05:08Z

auth/nginx.conf

+      ## See the map directive above where this variable is defined.
+      add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;
+
+      proxy_pass                          http://docker-registry;


Should this be docker.cdot.systems?

This should probably be http://registry:${PORT}

Yeah, we probably don't need the upstream and can use the docker network.

config/etc/nginx/conf.d

docker-compose.yaml

humphd

We should move toward testing this out.

Because you're using variables in the nginx conf, see the section "Using environment variables in nginx configuration (new in 1.19)" in https://hub.docker.com/_/nginx on how to pass these values in and have the container swap them for you (it won't happen automatically).

humphd · 2022-03-01T15:24:52Z

config/etc/nginx/conf.d

+
+    # SSL
+    ssl_certificate /etc/nginx/conf.d/domain.crt;
+    ssl_certificate_key /etc/nginx/conf.d/domain.key;


@manekenpix where will certbot put these?

We're setting /etc/letsencrypt below, but I don't know where under there it puts things.

This should probably be like what we do for telescope's front-end, here.
I think we'll have to change the docker file to match this.

humphd · 2022-03-01T15:25:33Z

config/etc/nginx/conf.d

+        return 404;
+      }
+
+      # To add basic authentication to v2 use auth_basic setting.


NOTE: for testing on the box, we could remove this completely. And when we do token-based/oauth, we won't need it.

humphd · 2022-03-01T15:26:30Z

config/registry-config.yaml

+    certificate: /path/to/x509/public
+    key: /path/to/x509/private
+    letsencrypt:
+      cachefile: /path/to/cache-file


This section seems wrong. Do we need to do any SSL stuff in here, since nginx is handling it? I doubt it.

manekenpix · 2022-03-01T23:55:49Z

config/registry-config.yaml

@@ -0,0 +1,41 @@
+# https://docs.docker.com/registry/configuration/


Where is this file used? it seems we're not passing its path as argument anywhere.

What's the answer to this question? Can we remove it? If it's going to be used it would be in the docker-compose file below, but you have it commented out.

I was unsure of the volume naming convention for it, and it would seem that the default configuration is adequate for what we have so far. I'm not sure this config file is needed?

The only thing we really need from this is to configure auth. Before we can use this for real, we need to have auth. In a follow-up we need:

GitHub-based auth, https://github.com/cesanta/docker_auth/blob/main/examples/reference.yml#L122-L163

Probably a Redis instance added to docker-compose.yml and configured to cache the tokens

@TDDR delete this file until we need it.

TDDR · 2022-03-03T02:19:28Z

After meeting with @manekenpix tonight this should be a testable version of the initial draft. I say should be because I ran out of time to test it out in the box tonight.

humphd

Have we tested this yet? I'd love to go into next week with this (mostly) ready to land.

humphd · 2022-03-04T23:51:48Z

config/etc/nginx/nginx.conf

+
+  server {
+    listen 443 ssl;
+    server_name docker.cdot.systems #${DOMAIN};


Let's remove the comment, and just hard-code this value. It's not going to change often enough to bother with the extra machinery to rotate it based on env vars.

humphd · 2022-03-04T23:52:45Z

config/registry-config.yaml

@@ -0,0 +1,41 @@
+# https://docs.docker.com/registry/configuration/


What's the answer to this question? Can we remove it? If it's going to be used it would be in the docker-compose file below, but you have it commented out.

docker-compose.yaml

TDDR · 2022-03-05T01:20:43Z

@humphd looks like I was testing it out as you where reviewing. The version that is in here now has been tested and it works

I can either work on resolving the final few comments possibly Saturday or Sunday. Or, maybe we could land what's here and make some issues of the remaining comments?

humphd · 2022-03-05T15:39:28Z

config/etc/nginx/nginx.conf

+
+  server {
+    listen 443 ssl;
+    server_name docker.cdot.systems; #${DOMAIN};


Remove the comment at the end of the line.

humphd · 2022-03-05T15:50:13Z

config/registry-config.yaml

@@ -0,0 +1,41 @@
+# https://docs.docker.com/registry/configuration/


The only thing we really need from this is to configure auth. Before we can use this for real, we need to have auth. In a follow-up we need:

GitHub-based auth, https://github.com/cesanta/docker_auth/blob/main/examples/reference.yml#L122-L163

Probably a Redis instance added to docker-compose.yml and configured to cache the tokens

docker-compose.yaml

humphd · 2022-03-05T15:51:17Z

docker-compose.yaml

+    image: registry:2
+    volumes:
+      - /mnt/docker0storage/registry:/var/lib/registry
+      #- ./config/registry-config.yaml:/config/registry-config.yaml  Untested use of the registry-config.yaml file


Let's remove this if not being used instead of commenting out.

humphd

One change and this is good to go!

humphd · 2022-03-08T15:26:54Z

config/registry-config.yaml

@@ -0,0 +1,41 @@
+# https://docs.docker.com/registry/configuration/


@TDDR delete this file until we need it.

humphd

Great. File the follow-up issues to do the rest, and this is good to land from my perspective.

humphd · 2022-03-09T02:02:22Z

@TDDR let's merge this, first of many!

Initial draft

ce3b53a

humphd requested review from DukeManh, Kevan-Y, manekenpix and humphd February 27, 2022 17:41

TDDR closed this Feb 27, 2022

TDDR reopened this Feb 27, 2022

humphd requested changes Feb 27, 2022

View reviewed changes

First update after review

a6b8277

TDDR requested a review from humphd February 27, 2022 22:21

TDDR force-pushed the initial-docker-compose branch from d76c9e8 to 9cb3dbc Compare February 27, 2022 22:27

Added certbot to docker-compose

32605b0

TDDR force-pushed the initial-docker-compose branch from 9cb3dbc to 32605b0 Compare February 27, 2022 22:28

humphd requested changes Feb 28, 2022

View reviewed changes

manekenpix reviewed Feb 28, 2022

View reviewed changes

Kevan-Y reviewed Feb 28, 2022

View reviewed changes

DukeManh reviewed Feb 28, 2022

View reviewed changes

humphd requested changes Mar 1, 2022

View reviewed changes

config/etc/nginx/conf.d Outdated Show resolved Hide resolved

docker-compose.yaml Show resolved Hide resolved

Second round of reiew corrections

f25c5ef

TDDR force-pushed the initial-docker-compose branch from 8fd80bf to f25c5ef Compare March 1, 2022 15:19

humphd requested changes Mar 1, 2022

View reviewed changes

manekenpix reviewed Mar 1, 2022

View reviewed changes

humphd requested changes Mar 4, 2022

View reviewed changes

Testable version of initial draft

8abbeb5

TDDR force-pushed the initial-docker-compose branch from 5319817 to 8abbeb5 Compare March 5, 2022 01:03

TDDR requested review from humphd and DukeManh March 5, 2022 01:28

TDDR requested review from manekenpix and Kevan-Y March 5, 2022 01:28

humphd requested changes Mar 5, 2022

View reviewed changes

TDDR requested a review from humphd March 8, 2022 01:21

humphd requested changes Mar 8, 2022

View reviewed changes

Initial version of registry files

8231325

TDDR force-pushed the initial-docker-compose branch from 2ba7b21 to 8231325 Compare March 8, 2022 16:20

TDDR requested a review from humphd March 8, 2022 16:23

humphd approved these changes Mar 8, 2022

View reviewed changes

manekenpix approved these changes Mar 8, 2022

View reviewed changes

TDDR merged commit b936d5f into DevelopingSpace:main Mar 9, 2022

		@@ -0,0 +1,41 @@
		# https://docs.docker.com/registry/configuration/

Initial draft #2

Initial draft #2

Conversation

TDDR commented Feb 27, 2022

humphd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kevan-Y Feb 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

humphd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TDDR commented Mar 3, 2022

humphd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TDDR commented Mar 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

humphd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

humphd left a comment

Choose a reason for hiding this comment

humphd commented Mar 9, 2022

Kevan-Y Feb 28, 2022 •

edited

Loading