.bazelrc generated range.sh credential helper invalidates other types of authorization at the host level

[sfc-gh-ptabor]

We discovered (thanks to @danielszot) that there is serious design flaw in range.sh approach to partial file-fetches.
It impossible to use on a single host (even different URLs) both range.sh .bazelrc solution and any form of credentials authorization. In particular from single service providing both: docker containers and *.apk packages, we were not able to serve them both.

What happens.

1. rules_apko generates .bazelrc file that install's credential helper that for each URL that ends with #_apk_range_bytes_ adds additional header:

Range: {from}-{to}

2. Credential helper is registered for entire host (domain or family of domaims). You cannot bind it to more specifc path like:
   my.artifacts.host.com/alpine_repository/... (https://github.com/bazelbuild/proposals/blob/main/designs/2022-06-07-bazel-credential-helpers.md)

3. Bazel tries to avoid making too many requests to credentials helper, so caches (at the bazel-server level) credentials for URL that is being fetched. For consecutive requests (multiple build for the same target), the credentials are provided directly from cache.

4. If the credential helper is installed, the Headers/Credentials/Auth passed directly to the bazel.download call are ignored.
   Credential helper takes precedence, even if it returns empty output and wipes Authorization.

5. The artifacts that need authorization (from the host where credential helper is used) cannot be fetched.

---

Solutions:

1. rules_apko stops using credential_helper. E.g. it can download the whole artifact and checksum whole artifact without doing partial fetches. Usage of credential_helper to override not-credentials headers is not supported by bazel.
   Would like to learn more why we do partial fetches: @thesayyn

2. rules_apko stops using bazel's downloader, but starts using custom download implementation like this one from rules_oci:
   No credential helper's complexity needed any longer on the user's side.

3. We expand Bazel protocol of credential_helper to be able to signal keeping original creds, rather than override. [
   @Yannic @tjgq @Wyverald @bergsieker PTAL] i.e.
   a) if credential helper will return empty value (empty set of hearders) -> use the original creds [this can be breaking change]
   b) if credential helper set's in GetCredentialsResponse a bit -> the original creds are used (output gets `merged').
   c) original credentials are passed as part of GetCredentialRequest [this might be security risky as allows credentials to escape]

4. We allow to bind credential_helper to a specific parent URL rather than whole domain:
   The current code is here.
   So we would accept e.g. following syntax:

common --credential_helper="myartifacts.company.com/repositories/alpine=%workspace%/.apko/range.sh"
or
common --credential_helper="*.myartifacts.company.com/repositories/alpine=%workspace%/.apko/range.sh"

@imjasonh @alexeagle @imjasonh FYI

[thesayyn]

Credential helper is a temporary solution to the problem of being able to "send arbitrary headers" with HTTP request which i already fixed upstream. credential helpers won't be necessary soon.

As for partial fetches, apks consist of three segments,

signature -> not guaranteed to be stable as signatures might change for various reasons. (eg, if the alpine private key gets leaked)
control -> guaranteed to be stable
data -> guaranteed to be stable

to avoid hard fails, in case the signature change, we only cache control and data and let signature to be fetched as needed. that's the reason why we don't specify checksum for signature fetches.

[thesayyn]

bazelbuild/bazel#19501

[sfc-gh-ptabor]

Thank you for explanation @thesayyn . Will wait for Bazel 7.1.0.
If that will take too long to get public, I would vote for solution number 2 as a stop-gap.

Closing.