[BUG] Docker registry rewriting not supported because of old types

[rassie]

What did you do

Trying to use the following registries.yaml to rewrite DockerHub images to our Gitlab's dependency proxy:

mirrors:
  docker.io:
    endpoint:
      - git.mycompany.com:443
    rewrite:
      ^(.*)$: /mycompany/dependency_proxy/containers/$1

After creating a cluster with this configuration, rewrite is silently missing from /etc/rancher/k3s/registries.yaml.

What did you expect to happen

rewrite should still be in /etc/rancher/k3s/registries.yaml. The reason for this is that Mirror struct is copied from an old version of k3s, which does not contain Rewrites, while k3s itself has moved to using https://github.com/rancher/wharfie/, which contains Rewrites.

To be fair, K3d documentation doesn't explicitely reference rewrites as being supported, but it does mention the file being "a regular k3s registries configuration file", which probably should count for something.

Which OS & Architecture

arch: x86_64
cgroupdriver: cgroupfs
cgroupversion: "1"
endpoint: /var/run/docker.sock
filesystem: extfs
name: docker
os: Ubuntu 20.04.5 LTS
ostype: linux
version: 20.10.18

Which version of k3d

k3d version v5.4.6
k3s version v1.24.4-k3s1 (default)

Which version of docker

Client: Docker Engine - Community
 Version:           20.10.18
 API version:       1.41
 Go version:        go1.18.6
 Git commit:        b40c2f6
 Built:             Thu Sep  8 23:11:45 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.18
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.6
  Git commit:       e42327a
  Built:            Thu Sep  8 23:09:37 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.8
  GitCommit:        9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[maxsargentdev]

Any maintainers know how much effort it would be to add this? Happy to do the work given a bit of direction.

I want to use the rewrite functionality to rewrite image tags pulled via a k3d pullthrough proxy so that they appear in the cluster as being direct from the source (my orgs artifactory instance), I can just pull them from the source but want a working cache on my machine to make things as quick as possible.

[iwilltry42]

Hi @rassie & @maxsargentdev how are you including the custom configuration at the moment?
Are you additionally using the --registry-use or --registry-create options?

[rassie]

I'm using a config file with a registries.yaml mounted:

volumes:
[...]
  - volume: config/registries.yaml:/etc/rancher/k3s/registries.yaml
    nodeFilters:
      - all
[...]

Everything else is in the issue description (i.e. contents of registries.yaml

Edit: I'm not using an internal registry, as far as I'm aware.

[iwilltry42]

@rassie , thanks for the input!
FYI: You may as well use the config file option registries.config or the command line flag --registry-config -> The config file expands environment variables.

[maxsargentdev]

I include the custom configuration inline as a file.

I have made a change in my fork to allow the rewrite configuration to be added if you want to use it, small change really.

That being said it only works via the config merge, not by extending the simple config API.

[maxsargentdev]

Here you go: #1215

[rassie]

This has been released with k3d 5.5.0, I can confirm that containerd configuration is now updated with mirrors and rewriting. I don't have it working yet, but that's probably a Gitlab or a containerd problem, so this issue is done. Thank you @maxsargentdev and @iwilltry42!