Provide builtins.currentSystem as a input to flakes or make system a parameter

[Mic92]

Is your feature request related to a problem? Please describe.

Right now one has to explicitly define system in flake outputs.
The nix flake itself already comes with boiler code like this:

outputs = { self } : let
   # ...
   forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f system);
{
  defaultPackage = forAllSystems (system: self.packages.${system}.nix);;  
};

I think having to use that much code hurts portability of the ecosystem because people
will only specify the minimum and there seems no easy way of overriding it without changing the
flake itself.

Describe the solution you'd like

outputs = { self, system } : {
  defaultPackage.${system} = self.packages.${system};  
};

Describe alternatives you've considered

Use boilercode in every project or rely on external libraries like https://github.com/numtide/flake-utils

If someone needs to explicitly specify platforms i.e. to build packages for different architectures with hydra this should be still possible.

[Mic92]

cc @zimbatm who is involved in flake-utils

[edolstra]

Not passing in system (or any other arguments) is intentional:

• It breaks hermetic evaluation: it makes the evaluation result depend on something external.
• It makes it impossible to enumerate the contents of the flake, since Nix doesn't know what the permissible values of the arguments are.
• The packages output should provide packages that work, not packages that might work.
• Evaluation caching becomes harder (not a huge problem for system, but would be a problem for arbitrary arguments).
• It would mean that fully-qualified flake output attributes no longer uniquely determine their evaluation result. I.e. an output attribute foo.bar will produce a different result on every system. Having a single name to specify flake outputs (rather than a tuple (attrname, system type, args)) is very useful, e.g. for Hydra job names (you can count on hydraJobs.foo to produce the same result for everybody).
• With a top-level system argument, you cannot use flake outputs for different systems in the same evaluation. (E.g. on x86_64-linux, you might want to use some i686-linux derivations like wine; and a Hydra aggregate job needs to be able to depend on jobs for different systems.)

[zimbatm]

There are multiple usability issues with the current design:

The list of supported systems is currently internal to the flake. As a flake author, it is useful to be able to specify which systems are supported. As a flake user, it means that it is not possible to change that list without forking the repository. Especially once the flake ecosystem starts to grow, it will quickly become prohibitive to introduce new platforms.

The amount of boilerplate that is needed to specify all the permutations of systems for all the derivations. That's why I wrote flake-utils, to make that usage a bit less cumbersome. The outputs are mixing both system-dependent and independent keys so it's not as easy as passing the system as an argument.

Typing x86_64-linux is error-prone. When a typo happens, the feedback system is not able to point to the typo directly. The user will see that their system is not supported, and then have to make the mental connection to look for a typo.

Each flake ends-up re-initializing nixpkgs. This adds to the boilerplate and evaluation time.

---

One possibility would be to introduce a new systems attribute to the top-level that lists all the supported systems by the flake. This has the advantage of making that list discoverable with tooling. And introduce typo checks for common archs. Then split the outputs to generic and system-dependent:

{
  description = "Usability flakes";
  systems = [ "x86_64-linux" "x86_64-darwin"];
  inputs = {}; # nothing changes here

  # system-independent outputs
  exports = inputs: {
    nixosConfiguration = {};
    overlay = final: prev: {};
    lib = {};
  };
  
  # system-dependent outputs
  outputs = system: { self, nixpkgs, ... }@inputs:
  let
    # I don't have a good idea to avoid that boilerplate
    pkgs = import nixpkgs {
      # Notice that the overlay is accessible via `self`
      overlays = [ self.overlay ];
      inherit system;
    };
  in
  {
    packages.hello = pkgs.hello;
    apps = {};
  };
}

One of the use-case that this design prevents is combining the derivations of multiple systems. This is for example if you want to create a release tarball that includes multiple architectures. In practice, that kind of construct is quite difficult to build outside of CI systems which have all the archs attached as remote builders.

[zimbatm]

Another possibility is that systems could inherit the list from the nixpkgs flake if left unspecified. That way, if nixpkgs gain a new architecture it will be easier to upgrade all of the flakes.

[7c6f434c]

Could we recognise the usefulness of ternary logic known-good/known-broken/unsure? The less overhead there is in checking whether something never-tried works due to upstream efforts, the better information we will have available.

[DavHau]

The original RFC states:

Flakes provide an attribute set of values, such as packages, Nixpkgs overlays, NixOS modules, library functions, Hydra jobs, nix-shell definitions, etc.

Is my assumption correct, that the idea of exposinglibrary functions has been abandoned? Since it's not possible to pass any arguments, it will never be possible to use a function of a flake?

I'm looking for exactly that use case. I'd like to expose a function that produces a python derivation, given a set of constraints.
Currently I have to use a really weird hack where I generate an infinitely deep attribute set, to allow the user to select packages:

nix run github:davhau/mach-nix#with.requests.tensorflow.aiohttp

With this trick, it's already possible to build constructs which allow passing arguments (kind of). Why not support it properly?

[edolstra]

No, you can definitely have library functions as flake outputs. You just can't call them from the command line.

Why not support it properly?

See my comment above.

Having said that, the configs branch does add experimental support for setting Nix module options from the command line (dc4a280).

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flakes-with-unfree-licenses/9405/1

[MagicRB]

I have similar problem as @DavHau, I'd like to give users of my flake the ability to compile firmware from the command line. The ideal UX is that one would write a config file, describing the build for the build system of the original project and then a derivation would be built, which would contain just the resulting .bin. This is currently impossible with flakes as I can't pass arguments into my flake from the command line.

What's the alternative?

[zimbatm]

What format is the config file in? nix run github:user/repo#compile --config ./myconfig.toml would work fine for example. nix run and nix develop use the current system to find the attribute. If the user wants to compose Nix files then they need another flake.

[MagicRB]

it's Kconfig, the kernel like build config

[domenkozar]

I wrote a proposal without knowing about this issue.

@Mic92 pointed me to it later on and it's interesting to observe that I've come to the same conclusions.

https://www.notion.so/Flakes-system-as-an-input-instead-of-an-output-2d2cdef8eac2434a833d6faae15b35c0

[thufschmitt]

I strongly support the idea of making the system handling simpler − esp. given that 80% of the time people just don’t care about it.

I think the current design has a really good thing though, in that it’s conceptually very simple (the only part of the Nix codebase that has to know about the notion of a flake “system” is the cli shortcut that expands foo#bar into foo#packages.x86_64-linux.bar). So although it’s painful and verbose, it’s explicit and doesn’t have any hidden magic.

(commenting here on @domenkozar's doc because I can't comment inline on it)

Flake would still pass the system implicitly by default (as it already does today when selecting the output attribute) but it would be explicit in flake.nix inputs.
There's already --system XXX argument to nix command line that can be used to override the default value matching host.

I guess you'd need a way to override the system arguments of the input flakes too (and separately from --system) for building multi-arch systems (Edit: I see this is handled by still having a .{system} attribute in the flake outputs, but that makes things slightly weird as it means that the system becomes both an input and an output)

We would also need to add top-level attribute supportedSystems = [ ... ] that would allow building hydraJobs and checks for all systems.

Then that leaves the problem that “The list of supported systems is currently internal to the flake” (Edit: you use inputs.nixpkgs.supportedSystems = [ ... ]; in a code snippet, which seems to imply that this could be overriden. In that case it's worth pointing out explicitely).

It is hermetic, the outputs would depend on the system input. Flakes caching can take the system into the account when calculating the key cache.

Maybe it’s the case (need to think it a bit more), but this only holds as long as Nix can generate a system-independent lockfile (because it would be pretty bad if the lockfile couldn’t be shared between two different systems)

There could be --local-system argument to flake commands to restrict evaluation to only one system.

Alternatively, that could be the default with --all-system providing the current default.

That bit isn't really clear to me: If system is an input, then evaluating the flake will only evaluate it for the system that's given as input. Or do you mean that a flake would actually evaluate to something like map (system: outputs (inputs // { inherit system; }) supportedSystems?

You'll have to evaluate each input with a different system. That can now be done in parallel.

I don’t think it’s really easier to parallelize this than the current version. The blocker in both cases is that the evaluator isn’t reentrant.

[thufschmitt]

Maybe a (relatively) small change to solve the problem of systems being hardcoded in a flake (which might also solve other issues) would be to allow raw Nix values as input for a flake (this shouldn’t mess-up caching as long as these values end-up in the lockfile). Then we could have a convention that flakes accept a supportedSystems input − in which case it could be overriden without having to fork the flake.

That does in a way add yet-more-convention and doesn’t do anything for reducing the boilerplate so I’m not sure it’s a good idea as-it-is, but there might be something to build on top of that.

[domenkozar]

I strongly support the idea of making the system handling simpler − esp. given that 80% of the time people just don’t care about it.

Thanks for putting it so succinctly!

I think the current design has a really good thing though, in that it’s conceptually very simple (the only part of the Nix codebase that has to know about the notion of a flake “system” is the cli shortcut that expands foo#bar into foo#packages.x86_64-linux.bar). So although it’s painful and verbose, it’s explicit and doesn’t have any hidden magic.

That's because it pushes most of the questions to the writer of the flake. I consider that a con, because things have to be implemented over and over again.

I guess you'd need a way to override the system arguments of the input flakes too (and separately from --system) for building multi-arch systems (Edit: I see this is handled by still having a .{system} attribute in the flake outputs, but that makes things slightly weird as it means that the system becomes both an input and an output)

It wouldn't be an output, but you'd declare it as a setting for the input itself - evaluating the same input for different systems. I'll update the document to clarify this.

Then that leaves the problem that “The list of supported systems is currently internal to the flake”

It gives the space for warning the user that the system isn't supported by the flake, instead of getting an attribute error. That seems quite an improvement for the experience.

Maybe it’s the case (need to think it a bit more), but this only holds as long as Nix can generate a system-independent lockfile (because it would be pretty bad if the lockfile couldn’t be shared between two different systems)

Looking at the lockfiles (there's, unfortunately, no documentation about them), what does narHash consist of?

That bit isn't really clear to me: If system is an input, then evaluating the flake will only evaluate it for the system that's given as input. Or do you mean that a flake would actually evaluate to something like map (system: outputs (inputs // { inherit system; }) supportedSystems?

It would mean you can evaluate the flake for all given systems, for example on a CI to make sure everything evaluates.

I don’t think it’s really easier to parallelize this than the current version. The blocker in both cases is that the evaluator isn’t reentrant.

It's parallelizable because one needs to pass system as an input, so when evaluating for more than one system, that's one evaluation per system.

Maybe a (relatively) small change to solve the problem of systems being hardcoded in a flake (which might also solve other issues) would be to allow raw Nix values as input for a flake (this shouldn’t mess-up caching as long as these values end-up in the lockfile). Then we could have a convention that flakes accept a supportedSystems input − in which case it could be overriden without having to fork the flake.

That will relative flakes would really go a long way with modularity. I'd still make supportedSystem a special meaning for warnings, etc as mentioned above.

That does in a way add yet-more-convention and doesn’t do anything for reducing the boilerplate so I’m not sure it’s a good idea as-it-is, but there might be something to build on top of that.

Yes, this design change is "convention over configuration" applied to dealing with system. Why do you think more conventions is bad?

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/improving-flakes/12831/24

[Ericson2314]

I agree with the gist of this change. The larger issue though is Flakes is constantly lagging behind idioms in Nixpkgs that have been polished over the years. For example:

1. We don't just pass one system, we pass stdenv.{build,host,target}Platform which provided much more detailed information and supports cross compilation.
2. lib.platforms used to be not a list of systems (again, too crude) but an arbitrary predicate on the host platform, which allowed intent to be conveyed much more precisely. This was reverted because it broke some (IMO layer-violating) feature of Hydra or nix-env.

We could incorporate all this stuff in Nix for Flakes, but that leaves Flakes quite brittle when we can and do refine this stuff from time to time in Nixpkgs. Or maybe we should implement more of Flakes in Nix. Factoring out lib/ and such things to make a standard library of sorts so we still have some agility. But that that point, what are flakes? Just something to manage inputs / lockfile without much extra structure?

[edolstra]

The larger issue though is Flakes is constantly lagging behind idioms in Nixpkgs that have been polished over the years.

Maybe the nix UI, but not flakes, because...

Just something to manage inputs / lockfile without much extra structure?

That's basically correct. The flake file format doesn't impose a lot of requirements on what the output attributes of a flake are. Thus it can accommodate whatever idioms regarding systems/platforms we may come up with in the future. (This is also why we shouldn't have a system input attribute, because that would lock us into a particular idiom.)

OTOH, various tools can define "well-known" flake output attributes, e.g. nix build looks for packages.<name>.<system> by default, as a convenience. These can of course evolve, e.g. we could have a packages_v2 output that uses a different encoding for system types.

This was reverted because it broke some (IMO layer-violating) feature of Hydra or nix-env.

The semantics of those attributes are determined by the tools that operate on them. You can't just change the meaning of platforms and then complain that nix-env barfs :-)

[edolstra]

Looking at the lockfiles (there's, unfortunately, no documentation about them), what does narHash consist of?

Lock files are documented here: https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html#lock-files

It's parallelizable because one needs to pass system as an input, so when evaluating for more than one system, that's one evaluation per system.

The current hydra-evaluator is already parallel. Having to deal with a system input type would just make it more complex since it then has two axes of parallelism to deal with: the system type and the jobs for each system type.

[gilligan]

That's basically correct. The flake file format doesn't impose a lot of requirements on what the output attributes of a flake are. Thus it can accommodate whatever idioms regarding systems/platforms we may come up with in the future. (This is also why we shouldn't have a system input attribute, because that would lock us into a particular idiom.)

I understand what you are arguing for, but I am confused because to me it seems like this might at odds with the goals of nix flakes as declared in the abstract of the RFC

Abstract: This RFC proposes a mechanism to package Nix expressions into composable entities called "flakes". Flakes allow hermetic, reproducible evaluation of multi-repository Nix projects; impose a discoverable, standard structure on Nix projects; and replace previous mechanisms such as Nix channels and the Nix search path.

So "standard structure" is indeed only to be understood as "something to manage inputs / lockfile" as @Ericson2314 put it above?

[roberth]

@tobiasBora The proposed Nixpkgs "problem" infrastructure could perhaps be used for this purpose. It wasn't specifically design for this, but it seems that it could be made to work well outside Nixpkgs as well. It's currently in development at:

• check-meta.nix: Add package warnings nixpkgs#177272

[roberth]

I've renamed this issue to hint at an alternative that is less contradicting to the requirements suggested in the first response.

Suggestions have been made to solve this by

• Adding a function such as perSystem
• Making the collections of system based attrs behave more like a function
  • Using __functor, or
  • Proxy like abstraction for an attrset #8187 with the side note that this either stretches the definition of a valid attrset, or it is an insufficient solution that would still require all valid systems to be enumerated by the user

We can change this back to a discussion about a single solution if you prefer, but otherwise I think this issue is representative of roughly the requirements

• Users shouldn't have to specify a list of systems. Some flakes are just portable, even if they do have derivations
• Users should be able to build a flake for a niche or not so niche system that wasn't specified in a flake they depend on. (Similar to the previous requirement, yet different)
• Cross compilation should be possible in a standardized way

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/experimental-does-not-mean-unstable-detsyss-perspective-on-nix-flakes/32703/2

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/experimental-does-not-mean-unstable-detsyss-perspective-on-nix-flakes/32703/25

[alper]

esp. given that 80% of the time people just don’t care about it

I would love never having to lookup the correct string for my system ever again (an absurd bit of busywork). Whatever you need to do to make this happen, go ahead.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-09-25-nix-team-meeting-minutes-89/33489/1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flake-design-of-system-os/36654/4

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flake-design-of-system-os/36654/7

[JumpIn-Git]

I did this in my flake, would this work?:

  outputs = {
    nixpkgs,
    home-manager,
    system ? builtins.currentSystem,
    ...
  } @ inputs: let
    <some stuff>
    pkgs = import nixpkgs {
      inherit system;
      config.allowUnfree = true;
      overlays = [import ./overlays {inherit inputs;}];
    };
  in 

I can't build it since I am still making my flake, could someone test
Edit: Nevermind, checking gave a error

[roberth]

If system becomes a parameter, it should not be a parameter for every output attribute, because that "harms"¹ attributes that don't use the parameter, or use a different set of parameters. This includes attributes like overlays, nixosConfigurations, lib, modules, nixosModules and more - quite common.

¹: harm => ability to use the wrong variable + the usual reasons for Interface Segregation. Example: accidental use of the "impure" system causes things like remote deployments to different systems to fail.

[JumpIn-Git]

Would something like this work?:

  outputs = { nixpkgs, ... }@ inputs: let
    system = nixpkgs.stdenv.hostPlatform.system;
    # or
    #system = nixpkgs.system;
  in

[infinisil]

@JumpIn-Git Sorry to be a bit crude, but please be more respectful of people's time and attention. This thread is for discussing development of Nix, with many busy people subscribed to it, making your untested support-level questions not a great fit. For somebody relatively new like you I suggest the main Nix/NixOS Matrix room instead, but even there you're expected to spend more effort before asking, among other common etiquette.

[JumpIn-Git]

@JumpIn-Git Sorry to be a bit crude, but please be more respectful of people's time and attention. This thread is for discussing development of Nix, with many busy people subscribed to it, making your untested support-level questions not a great fit. For somebody relatively new like you I suggest the main Nix/NixOS Matrix room instead, but even there you're expected to spend more effort before asking, among other common etiquette.

huh? what i am asking is related to this subject; those values give the current system

[eclairevoyant]

@JumpIn-Git This thread is not a support forum, please ask on discourse.nixos.org or the mentioned matrix room.

Anyway to get back on topic about the requirements:

• Users shouldn't have to specify a list of systems. Some flakes are just portable, even if they do have derivations
• Users should be able to build a flake for a niche or not so niche system that wasn't specified in a flake they depend on. (Similar to the previous requirement, yet different)
• Cross compilation should be possible in a standardized way

Ultimately IMO this points to the need for changing the structure of (currently system-dependent) flake outputs. Instead of <output>.<system>.<name>, which frankly didn't make much sense overall, since 99% of the time derivations are going to be the same across systems, the structure should be more like <output>.<name>. This would also address @edolstra's concern about getting locked into certain idioms if we sidestep the need to get locked into a specific way of supplying system.

Instead, couldn't we leave the tasks of specifying and constraining systems to nixpkgs? We already have meta.*platforms for this purpose, I don't think we even need to wait for RFC 127. That should also make it much easier to handle cross-comp as well. Though we can still improve the cross-comp process in nixpkgs, we are more flexible to address this in nixpkgs than in nix, especially once the default flake schema is stabilised and therefore set in stone.

And more to the point, why should nix concern itself with what system it's running on?

• It breaks hermetic evaluation: it makes the evaluation result depend on something external.
• It would mean that fully-qualified flake output attributes no longer uniquely determine their evaluation result. I.e. an output attribute foo.bar will produce a different result on every system.

I disagree, or in any case, we are already in that situation today since we're picking the system presumably out of thin air from the CLI. Whereas in consuming flakes, it makes sense to ensure that system is effectively passed to the derivation via the same mechanisms which we control nixpkgs instances' platforms today.

With a top-level system argument, you cannot use flake outputs for different systems in the same evaluation. (E.g. on x86_64-linux, you might want to use some i686-linux derivations like wine; and a Hydra aggregate job needs to be able to depend on jobs for different systems.)

I don't have a good answer for this one, but I think we should be able to come up with something less clunky than the current outputs structure...

[Mic92]

huh? what i am asking is related to this subject; those values give the current system

I don't think it was a support question, but unfortunately you cannot use system from nixpkgs because nixpkgs needs system from the flake in the first place. So this is a catch22.

[eclairevoyant]

Sure, provide it as an arg when you import nixpkgs. Or if you're using the CLI, then the option to provide it automatically should be made available. But either way, my point was that it shouldn't be part of the attr path.