Optional client side building

[Ericson2314]

Is your feature request related to a problem? Please describe.

1. For fixed output derivations that need authentication, it is probably better to run them as the current user in order to give them secrets, especially ephemeral secrets (like expiring tokens) that might require some humans in the loop (various 2fa schemes) and are cumbersome to store in the store.

2. Builtin fetching should be representable by derivations #9077 once the above is sorted out, we should do this too. Currently the main reason fetching is not done with derivations is authentication. This provides a proper solution. All fetching done as client-side derivations nicely meets in the middle of the current fixed-output derivations vs libfetchers divide.

3. General decoupling. Building shouldn't depend on using the SQL database (and I don't think it currently does). It ought to work with other stores that also provide a file system view (LocalFSStore).

Describe the solution you'd like

1. Have a method getBuilder like getFSAccessor, moving building methods there.

2. Remote stores now can either use the current remote side scheduling/building or do their own. This can be dynamic, unlike strategies that bake the choice into the Store class hierarchy.

3. LocalDerivationGoal should merely assume LocalFSStore, not LocalStore.

4. LocalStore should be renamed SQLiteStore.

This provides enough building blocks to allow for a wide variety of remote building configurations.

Additional Context

Talking to @kolloch about AWS fetching and also the ACL store RFC.

Priorities

Add 👍 to issues you find important.

[Ericson2314]

2023-11-17 Nix team meeting notes:

• @thufschmitt: Sounds complex
• @Ericson2314: No necessarily that much. The build logic is already mostly independant from the store

Deferred, waiting for John and Tom to write a more advanced desing doc. Sounds good in principle.

[kolloch]

Heyo @Ericson2314, thanks for this, some questions:

1. How would this look for the user? Specify a certain derivation attribute to force local building?
2. Would that still allow paralelization? How is that managed?

[kolloch]

3. General decoupling. Building shouldn't depend on using the SQL database (and I don't think it currently does). It ought to work with other stores that also provide a bike system view.

s/bike system view/build system view/?

I cannot follow this and the implementation details. What is the relevance of SQLite in this context?

[Ericson2314]

How would this look for the user? Specify a certain derivation attribute to force local building?

Yes can rig up something with "required system features" I assume.

Would that still allow paralelization? How is that managed?

Yes. In the same way it is today, with or without remote builders.

I cannot follow this and the implementation details. What is the relevance of SQLite in this context?

Ooops I meant "file system view". Thanks for pointing out.

SQLite is not relevant! That is my point. However the code today currently requires building to happen with a LocalStore, i.e. a SQLiteStore (as it should be called). This is an artificial restriction that just confuses things.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-11-17-nix-team-meeting-minutes-104/35753/1