devos-as-library

[blaggacao]

Proposed API (v0.2)

# flake.nix
{
  inputs.devos.path = ./lib;
  # inputs.devos.url = "github.com:divnix/devos?dir=lib";
  outputs = inputs@{self, devos}: let name = "mydevos"; in
  devos.mkFlake {
    inherit self inputs name;
    hosts = ./hosts; # or attrset, usable also: devos.lib.attrsFromPath
    profiles = ./profiles; # or attrset
    modules = [ ]; # or path
    suites = { }; # or path
    packages = name: final: prev: { }; # name spacing external pkgs sharing | or path
    "..." # v0.2 place holder ;-)
  }
  //
  {
    # extend with custom stuff
    hmConfigurations.some = devos.mkHmConfiguration {};
    nixosConfigurations = lib.mapAttrs (n: v: {name = n + "-staging"; value = v;}) (devos.lib.mkAttributesFromPath ./staging);
    devshellModules = {}; # prototyping / showcasing future ideas for devos-as-library
  };
}

Rationale

(tbd — will redact later)

---

Related / Affected / Solvable

• redesign before flakes stabilize: 1.0 #121 #36
• #91
• 1.0 Roadmap #121
• Expected that not all packages are exported? #151
• fix lib docs #166
• Simplify devos #179
• name space ./pkgs overlays #183

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[blaggacao]

To complement the collaboration model to somebody who wants to extend devos lib itself, we can recommend to do it as a patch / PR in the spirit (and anticipation) of NixOS/nix#3920

That encourages to surface community work and increases the amount of opportunities to generate consensus.

[Pacman99]

I like this a lot 👍 as I mentioned earlier, I would prefer it if we first actually create the library function for mkFlake and have that not implement a new api design and just implement the current api. And then once we finalize the devos api, we can do another PR to update mkFlake/evalArgs.

[Pacman99]

On the proposed api itself.
I was considering allowing paths, but then decided against it. It makes it harder to type check things within the module system and when we don't allow paths, we can add some other features. For example the current #206, actually lets you drop certain attributes within files like suites.userSuites. Because the suites apply key will just append an empty attrset for anything that you don't include. And that feature is much harder to implement if we allow paths.
So essentially you have to do an import ./suites if its not in the default location.

An inherit self has to be included too, there's no way to write mkFlake if it doesn't get the flake's self input.

Not really sure what to do with 'name', I feel like name is just the repo name and that doesn't need to be passed to mkFlake.

We should also decide on where to use auto-import magic. I'm fine with it as its used right now. hosts are auto-imported and have clear rules for how they are imported. Profiles are technically auto-imported, but because of how suites work, you still explicitly have to select suites, so I think thats ok.

[nrdxp]

We should also decide on where to use auto-import magic. I'm fine with it as its used right now. hosts are auto-imported and have clear rules for how they are imported. Profiles are technically auto-imported, but because of how suites work, you still explicitly have to select suites, so I think thats ok.

This was the intention. I don't know why, but selecting profiles to use directly by path felt kind of like a hack to me. Even though the attribute values are really just the very same path, it makes the api very explicit for profiles, and doesn't confuse their usage with other imports, such as new module options. I also didn't want to enforce this for users who may just prefer the workflow of managing imports the old fanshioned way.

The auto imports for hosts was originally conceived for two reasons:

1. Flakes were still brand new at the time, and I wanted to conceptually promote back compat by allowing users to treat each individual nix file under hosts exactly as they would a configuration.nix.
2. Enforcing the setting of hostName by filename to allow for easier nixos-rebuild switching. It's a sane default to enforce, so you always know which system your working on.

Number one may no longer be an issue as users become more familiar to with flakes and their API.

Number two still seems like it's nice to have, but may be easily extended to include directories for more involved host systems.

As an aside, I would like to encourage upstream, at some point, to change the nixos-rebuild command from a perl script to a first class nix operation, such as nix os switch and nix os "devos#hostThatDoesntMatchMyCurrentHostname" switch. This way, users of nix can more easily build a NixOS system, even if they are not currently using one.

[Pacman99]

I'd say I'm pretty familiar with the flakes api now mostly due to working on this repo. But I still like the auto-import for hosts. I think its nice to have a folder where all hardware/server specific information goes. And I would personally prefer it to be auto imported with a specific file system hierarchy.
So heres a proposal:
Allow for a standard way to represent a file system hierarchy through attrsets. So hosts can either be files within hosts where the name is automatically the filename with .nix stripped or directories with default.nix and the name is the folder name. Or if you don't want to use auto-import magic, you can pass an attrset that maps names to files. This should also cover profiles and sub profiles.

profiles:
minecraft/default.nix
matrix.nix
matrix/bridges.nix

could map to

{
  minecraft = {
    default = ./minecraft;
  };
  matrix = {
    default = ./matrix.nix;
    bridges = ./matrix/bridges.nix;
  };
}

The api could probably use some work, maybe not have the default or enforce more restrictions like they have to be folders.

[nrdxp]

I was thinking whatever the end result that it should be used for both profiles and hosts, accentuating that hosts are essentially a subset of profiles with more specific information. That way, if users have questions about the auto-import logic, they only have to reference a single libary function.

In a previous iteration of DevOS (essentially my personal dotfiles of stable nix) I had a similar auto import feature for speed of development, where I excluded any nix files that started with _. So if a nix expression is auxillary and not meant to itself be a profile/host, it's made explicit by the leading _. Of course maybe a more descriptive prefix is in order.

We could also invert the logic and only import profiles that start with a special prefix, ignoring all other nix files besides default.nix in a directory.

[Pacman99]

Yeah 👍 on a standard auto-import magic function. But I think it would be good to accept both either a path or an attrset that can describe the hosts/profiles in a standard way. I guess thats only necessary if theres enough people that actively don't want auto-importing. So if most of the frustration for auto-import can be fixed by a standard function and some documentation then its unnecessary.

[nrdxp]

I guess thats only necessary if theres enough people that actively don't want auto-importing.

We might make it easier to extend nixosConfigurations with custom nixos modules. We could essentially map a call to devosSystem over an additional attribute set where users can set host modules in a more manual fashion and then run recursiveUpdate. This would provide a workaround for #117 as well.

[RealityAnomaly]

If anyone is interested in a reference for something similar, I wrote a flake inspired by devos that can actually be inherited by other flakes as a personal experiment (and extensibility is as easy as mkIntermediateArnixRepo / mkTopLevelArnixRepo):

https://github.com/ArctarusLimited/arnix - the base
https://github.com/CitadelCore/nixflk - top-level

[blaggacao]

Not really sure what to do with 'name', I feel like name is just the repo name and that doesn't need to be passed to mkFlake.

An example community sharing model implies people not only using devos/community as base but also pacman99/main as input.

The user blaggacao/main is interested in pacman99/main.synapse, but it happens so that pacman99/main also defines a pacman99/main.kakoune package. However, the user wants to keep his own blaggacao/main.kakoune implementation.

Hence it appears to me that those use cases can only be solved with reasonably unique name spacing:
pkgs.blaggacao.kakoune vs pkgs.pacman99.kakoune.

There is still a twist to this, for ease of use and back-compat, blaggacao could just reference his kakoune through pkgs.kakoune while pacman99's could be accessed through pkgs.pacman99.kakoune.

So essentially you have to do an import ./suites if its not in the default location.

Not too bad 😄 Although we might just run them through a maybeImport before evaluating them with the module system. I think it's worth, since this is becoming essentially a table of content. import is not necessary for human understanding.

An inherit self has to be included too, there's no way to write mkFlake if it doesn't get the flake's self input.

I see, because it accessors outputs of self that is. And also probably inputs.

[Pacman99]

Interesting I think I see the use case. But I would generally prefer the alternative, others can access my packages with inputs.pacman99/pkgs.pacman99. But I only have to do pkgs. And nix flakes already creates some form of namespacing when you add an input..

I gues this would not work great with modules that also need custom packages. In that case you would have to import the module, and then in extern(or whatever the alternative would be) you can add an overlay to add that custom package. So if devos handles namespacing, then you could just add the overlay and then all my modules would work.

So I'm not against namespacing and I see the benefits. But we should be careful to maintain a balance between convenience(easily accessing your own packages) and sharing(easy to export and import other peoples code).

So essentially you have to do an import ./suites if its not in the default location.

Not too bad smile Although we might just run them through a maybeImport before evaluating them with the module system. I think it's worth, since this is becoming essentially a table of content. import is not necessary for human understanding.

With the current mkFlake PR this is no longer true, you have to import the file. Since there is no longer a default location. But yeah since flake-utils already has the function, we could do that. Actually I still don't want to allow paths because that hurts type checking, now we have to allow path as a type for a lot of the options.

[blaggacao]

I gues this would not work great with modules that also need custom packages. In that case you would have to import the module, and then in extern(or whatever the alternative would be) you can add an overlay to add that custom package. So if devos handles namespacing, then you could just add the overlay and then all my modules would work.

Great point. Havn't thought of that. So it looks like the most straight forward solution is to indeed treat the devos community as a global unique namespace and namespace properly all the way through.

As a side effect, this also asserts awareness that everyone acts "as a (good) devos citizen in a global namespace".

The thought is not mine, btw. I just derived it from how flake-utils.simpleFlake is designed in a holistic way.

[Pacman99]

@blaggacao Could you actually create this as a PR to add an API.md or something to doc. That way we can create threads for specific parts of it and request changes. Soft of like nixos rfc process. It doesn't have to actually be merged anytime soon, it would just be easier to discuss with.

[nrdxp]

A central part of devos as a library is going to require some additional convenience added to the current template.

One might think, at first, that the interface to templates restricts derivations since it only accepts a path, until you remember lazy linkage to outPaths. We could, therefore, add a build stage to the template process and perhaps incooperate #16 to allow for customized users and basic develop and graphical profiles.

We could further fenangle arguments to the builder. Although this would be only from the source because flakes' lack of arguments, this is a template so users are expected to have a copy anyhow.

This would give us simple custimization of the template itself as various template outputs.

[Pacman99]

I like the idea, I think it would really improve on boarding for new users. And maybe we could allow users to mix and match things from community and core, so you can pull the 'community profile' template while using the 'core modules' template.
One thing to note is that it would add more complexity to this repository, since we would have to maintain more templates - totally worth it. So perhaps we should have the discussion soon on extracting the devos library to a separate repository. So this repository can be focused on the templates and sharing, while the other repository is focused on cleaning up the api and adding/improving features.

[blaggacao]

While prototyping on blaggacao#10, I also realized that mkFlake api needs to be complemented by a sound devos-lib api. Just wanted to log in on the record in here.

[blaggacao]

@nrdxp

This would give us simple custimization of the template itself as various template outputs.

I figured the jobs output would not be the worst location to implement a templating action.

Not exactly sure how that would be invoked with nix, but I guess we have flk for those kind of things.

• flk populate (say)
• prompt for some questions
• modify nix code mediated by builtins.unsafeGetAttrPos

Just an idea leveraging pure nix concepts.

[blaggacao]

This is implemented via api-next branch.

In an effort to clean up issues, I'm going to close this.