libstore: Add the ability to use different auth methods with Http Binary Caches

[georgyo]

Motivation

Right now the only auth methods allowed with http caches was the default basic auth method. This meant if you wanted to have any other mechanism for authentication you requiring patching the code.

My personal motivation is enabling ?authmethod=negotiate.

Context

This PR is very similar to #10568 but made to be generic and support as many methods from libcurl as we can reasonably support.

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[georgyo]

@roberth Do you have any thoughts on this version of the PR?

[Ericson2314]

This should be an enum struct.

[georgyo]

Uploaded my attempt at this, however I don't write much C/C++ code.

I tried several different ways to make this a Setting<HttpAuthMethod> using the templates for BaseSetting, but I could not get this to work for a URL type parameters, and indeed none of the other url parameters are using enums.

[fricklerhandwerk]

Discussed in Nix maintainer meeting:

• @edolstra: this could done be in a more general way building on top of Pluggable authentication #9857
  • this change only enables authentication methods for the HTTP binary cache and nothing else
  • pluggable auth does not deliver the feature right now though
    • that other PR does two things actually, splitting it may make it easier to merge
• we'd merge this PR if it didn't introduce a setting that we'd have to support forever
  • this is risky given this topic is currently under development
• we recommend trying to implement the feature using pluggable auth, trying out the branch
  • a valuable contribution would also be to rebase the pluggable auth PR
  • another thing would be doing some research whether Git credential helpers can be used to get the authentication method

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-06-24-nix-team-meeting-minutes-155/47739/1

[georgyo]

An alternative approach to choosing the exact curl auth method would be to set to CURLAUTH_ANY, which is the git default. The pros here is that this will enable curl to upgrade the auth method to the most secure.

This is how Git behaves when talking to an HTTP repo and allows it to handle both Basic Auth and things like Negotiate. The downside to that approach is if the server advertise negotiation there is no way to force git to use basic.

Setting CURLAUTH_ANY would enable kerberos, (and digest/ntlm) support in nix without also introducing a flag that would have to be supported forever.