force passing ci tools version

[samansmink]

This PR aims to make calling the .github/workflows/_extension_distribution.yml workflow a little more predictable. It also fixes the CI failures that will happen whenever trying to build an extension on an old extension_ci_tools submodule after #79.

What this means is that all extension will now be required to pass both a duckdb_version and a ci_tools_version. These will in most cases contain the same value (the tag of the latest stable release of duckdb or main).

The reason we need this is because we want to checkout the extension_ci_tools submodule of the extension to the version of the called workflow. So for example in the following invocation of _extension_distribution.yml:

duckdb-stable-build:
    name: Build extension binaries
    uses: duckdb/extension-ci-tools/.github/workflows/[email protected]
    with:
      duckdb_version: v1.1.0
      extension_name: quack

What we would like to do automatically is to checkout the extension_ci_tools submodule to the v1.1.0 tag that we pass to the workflow. However, due to actions/toolkit#1264, this is actually ***ing difficult.

There are a few solutions here but all of them seem evil. The most sane way in my (and @carlopi's) opinion is to require explicitly passing the version as another tag, making it so that you need to pass the version 3(!) times:

duckdb-stable-build:
    name: Build extension binaries
    uses: duckdb/extension-ci-tools/.github/workflows/[email protected]
    with:
      duckdb_version: v1.1.0
      ci_tools_version: v1.1.0
      extension_name: quack

We have considered creating a single target_version input parameter but while nice in most cases, this seems confusing whenever you want to build for a specific commit of duckdb. Making everything explicit is the most clear and even though its ugly, its at least somewhat clear whats going on.

[carlopi]

I think we enumerated up to 5 alternatives to this, maybe for fun it would be cool to write them down explicitly, just to remember whoever passes / future us of why this is not that bad in the end.

[samansmink]

other options considered here are:

inlining everything in extension_ci_tools that is not a reusable workflow

Inlining complex docker files into yaml is horrible

making separate repository for files and workflows

Split out the dockerfiles into yet another repository and have its hash hardcoded into the workflow. This will work, but we also want the makefiles to be checked out to the right version and potentially any other files that we want to have available both in CI and when developing an extension locally

HEAD-1 commit trick

We could hardcode the hash of the last commit of this repo and use that to check out the submodule, this is just awful. This means that updating the dockerfiles will always require 2 commits, one to do the change, then one to update the hash.

Query the GH API to get the hash

there's actually an action https://github.com/dariocurr/checkout-called which can do this for you. It feels fragile to let our ci rely on manually querying the GH api so we prefer to just pass it explicitly