Allow overriding ruby source url during build time

[Earlopain]

This makes it possible to build against ruby master (as long as there are no additional changes that would need to be made)

I understand that as per #222 providing head images directly is not feasable. This makes it one step simpler to use these yourself if you want to. I believe it doesn't add too much complexity.

Due to the volatility of the nighly builds I added a build arg to skip the checksum check. It wouldn't really work that well otherwise.

A sample docker-compose.yml to demonstrate:

services:
  ruby:
    build:
      args:
        RUBY_VERSION: 3.4-dev
        RUBY_DOWNLOAD_URL: https://cache.ruby-lang.org/pub/ruby/snapshot/snapshot-master.tar.xz
        SKIP_CHECKSUM_CHECK: 1
      dockerfile: 3.3/alpine3.19/Dockerfile

It doesn't work for the 3.0 images since they have a different format but it's EOL anyways and soon to be removed I imagine.

[tianon]

Sorry, I'm generally pretty strongly -1 on changes that add codepaths we do not use/test ourselves (because we cannot make any promises about how well they will be supported and maintained over time).

In theory, you should be able to get this working by hand-crafting (probably via jq) a new entry in versions.json and running apply-templates.sh to have the existing templates put out some Dockerfiles that should build appropriately.

[tianon]

Oh baby, https://cache.ruby-lang.org/pub/ruby/snapshot/snapshot-master.json makes this even easier to implement because it matches the existing data format! 😂

[tianon]

$ wget -qO- 'https://cache.ruby-lang.org/pub/ruby/snapshot/snapshot-master.json' | jq -s '.[1][0] as $master | .[0] | .master = (.["3.3"] as $donor | { version: "master" } + $master + { variants: $donor.variants, rust: $donor.rust, rustup: $donor.rustup } | with_entries(if .key == "filename" then .key = "url" | .value |= map_values("https://cache.ruby-lang.org/pub/ruby/snapshot/" + .) else . end))' versions.json - > versions.new.json
$ mv versions.new.json versions.json
$ ./apply-templates.sh master
...
$ docker build --pull master/bookworm
...
+ ruby --version
ruby 3.4.0dev (2024-04-26 master b6489e9f62) [x86_64-linux]
...
Successfully built fb95e684415b
$ docker run --rm fb95e684415b ruby --version
ruby 3.4.0dev (2024-04-26 master b6489e9f62) [x86_64-linux]

and just that jq line again, but with more whitespace for better readability:

wget -qO- 'https://cache.ruby-lang.org/pub/ruby/snapshot/snapshot-master.json' \
	| jq -s '
		.[1][0] as $master
		| .[0]
		| .master = (
			.["3.3"] as $donor
			| { version: "master" }
			+ $master
			+ { variants: $donor.variants, rust: $donor.rust, rustup: $donor.rustup }
			| with_entries(
				if .key == "filename" then
					.key = "url"
					| .value |= map_values("https://cache.ruby-lang.org/pub/ruby/snapshot/" + .)
				else . end
			)
		)
	' versions.json - \
	> versions.new.json

Edit:

Example Generated JSON block:

  "master": {
    "version": "master",
    "date": "2024-04-25",
    "url": {
      "gz": "https://cache.ruby-lang.org/pub/ruby/snapshot/snapshot-master.tar.gz",
      "xz": "https://cache.ruby-lang.org/pub/ruby/snapshot/snapshot-master.tar.xz",
      "zip": "https://cache.ruby-lang.org/pub/ruby/snapshot/snapshot-master.zip"
    },
    "size": {
      "gz": 22438836,
      "xz": 16669328,
      "zip": 27541030
    },
    "sha1": {
      "gz": "d49b730ace5f70cbb104e2f230882222cf4e7a19",
      "xz": "1bfcfbb5942ee7f15d98123c741c742d4e324f47",
      "zip": "b10c7876e5a26ffb8736dc736881592538c0a527"
    },
    "sha256": {
      "gz": "1129f3c5a198bf194aa40ede4b2b436917c48cd9856738c7e3d5d23d4fca39a1",
      "xz": "9df2a5c7b899dfbf481954ef430d6424a04cf3f648324368a6d55a661aa73cfa",
      "zip": "490fbee80924d8943c03d1faf02a5eebf583c2ae8669f15711440c1be5822309"
    },
    "sha512": {
      "gz": "a38030d152c363b0800f3b3fc04044de8f9944f6d3c22a8934b51170cd42c8d07c7c71ca1cd314861ac00bac71dcc65bf25cadba7802a02d3c512c9401b07b51",
      "xz": "430750b5e551d1d9c1b932d1534006797b6a97a9422ff36db27a7da4da258bd38059f97f3ebf883f6b6fb075e1b84193a644e7cfe0522f6064f98e58088c8e1b",
      "zip": "ab90e4f8a61af34932c984edcc13e6adc5cc594a4e4471f9bc8e067eb110f1de7ab6f650af3bc8768fbe34006ca00a457dfa875c009c3c614020d9571a6764dd"
    },
    "variants": [
      "bookworm",
      "slim-bookworm",
      "bullseye",
      "slim-bullseye",
      "alpine3.19",
      "alpine3.18"
    ],
    "rust": {
      "version": "1.74.1"
    },
    "rustup": {
      "arches": {
        "amd64": {
          "glibc": {
            "arch": "x86_64-unknown-linux-gnu",
            "sha256": "0b2f6c8f85a3d02fde2efc0ced4657869d73fccfce59defb4e8d29233116e6db",
            "url": "https://static.rust-lang.org/rustup/archive/1.26.0/x86_64-unknown-linux-gnu/rustup-init"
          },
          "musl": {
            "arch": "x86_64-unknown-linux-musl",
            "sha256": "7aa9e2a380a9958fc1fc426a3323209b2c86181c6816640979580f62ff7d48d4",
            "url": "https://static.rust-lang.org/rustup/archive/1.26.0/x86_64-unknown-linux-musl/rustup-init"
          }
        },
        "arm64v8": {
          "glibc": {
            "arch": "aarch64-unknown-linux-gnu",
            "sha256": "673e336c81c65e6b16dcdede33f4cc9ed0f08bde1dbe7a935f113605292dc800",
            "url": "https://static.rust-lang.org/rustup/archive/1.26.0/aarch64-unknown-linux-gnu/rustup-init"
          },
          "musl": {
            "arch": "aarch64-unknown-linux-musl",
            "sha256": "b1962dfc18e1fd47d01341e6897cace67cddfabf547ef394e8883939bd6e002e",
            "url": "https://static.rust-lang.org/rustup/archive/1.26.0/aarch64-unknown-linux-musl/rustup-init"
          }
        }
      },
      "version": "1.26.0"
    }
  }

[Earlopain]

The version file is pretty neat, I wasn't aware that it exists. Switching it out is indeed very easy. I've opted to go with a fork that builds these images daily on GitHub actions and pushes them to their container registry. Here's the workflow I ended up with: https://github.com/docker-ruby-nightly/ruby/blob/f682f975d682defdf22becf475945fca25aab95e/.github/workflows/nightly.yml, I can then use the image like so ghcr.io/docker-ruby-nightly/ruby:alpine3.19-nightly-2024-04-25. I know in the linked issue there some thing that publishes as well but that doesn't do alpine.

It's missing a bunch I imagine, especially around the metadata I was very lazy. It just got a tag and that's it. Also not very multiarch friendly. But it's whatever, works for me.

Thanks again for your pointers on this. I'll go ahead and close this now.