Improve flake usability

[mkenigs]

Copying discussion from #3573. @edolstra is this something that needs to wait until flakes stabilize a bit more, or are there things I could work on starting to change? Can close this if it doesn't make sense at this point.

I've been thinking about usability, and I guess some of this might be more of my thoughts related to nix in general. But if I was trying to to convince someone to use nix, I would want a flake to look something more like this:

{
  description = "optional description";
  inputs.build = [ boost ];
  inputs.test = [ testframework ];
  outputs = { self, inputs }: {
    mkFlake {
      name = "name";
      build =
        ''
          make
        '';
      install =
        ''
          cp name $out/bin/
        '';
      run = "$out/bin/name";
      test =
        ''
          test
        '';
    }
  }
}

I don't know how things like nixpkgs, system, mkDerivation, and other outputs would be handled, so I guess this is more just my thoughts rather than something concrete. But if it was possible, I think something along these lines would be way easier to digest as a newcomer to nix. It's pretty easy to understand what's going on: you say what dependencies you need, nix does it's nix magic, and you get something you can build/run/develop/test anywhere with nix build/run/develop/test (flake check). I think it's hard to realize that's what nix can do when it's hidden behind complexity, but it's actually incredibly appealing once you see it.

Originally posted by @mkenigs in #3573 (comment)

@mkenigs Agree completely but that's out of scope for the first iteration of flakes. In the future we might have a flake.toml containing a high-level description of a package, without any extraneous cruft like mkDerivation, function calls etc.

Originally posted by @edolstra in #3573 (comment)

[mkenigs]

Somewhat related to #3843

[domenkozar]

It would be helpful to also provide the same example under the current flakes to better see how they compare.

[matthewbauer]

The configure / build / install phases are pretty specific to Nixpkgs. I am hoping Flakes would not have to know about Nixpkgs specifically. It happens quite a lot where there really isn't a true "configure" phase being run with newer build systems. Things like Bazel don't even really have a concept of "install" - it's all just building (I guess install would be just cp -r * $out).

[mkenigs]

@domenkozar not sure if I'm getting all this right, but I think it would be something like:

{
  description = "optional description";

  outputs = { self, nixpkgs }: rec {
    packages.x86_64-linux.name =
      with nixpkgs.packages.x86_64-linux;
      with nixpkgs.builders;
      with nixpkgs.lib;

      stdenv.mkDerivation {
        name = "name";

        buildInputs = [ boost ];

        src = self;

        # could leave buildPhase out since it's default
        buildPhase =
          ''
            make
          '';
        installPhase =
          ''
            cp name $out/bin/
          '';
        doCheck = true;
        checkInputs = [ testframework ];
        checkTarget = "test";
      };

    defaultPackage.x86_64-linux = packages.x86_64-linux.name;

    defaultApp.x86_64-linux = {
      type = "app";
      program = "${self.packages.x86_64-linux.name}/bin/name";
    }
    checks.build = packages.x86_64-linux.name;
  };
}

[matthewbauer]

I've been working on a more structured version of stdenv.mkDerivation for use in flakes. It's supposed to be pretty similar to how stdenv.mkDerivation works, but with some improvements. For instance, "custom" phases are disallowed

Here's an example of how it looks (with a slightly simplified version of the Nix flake):

{
  description = "The purely functional package manager";

  inputs.make-package.url = "github:matthewbauer/make-package.nix";

  outputs = { self, make-package }: make-package.makePackagesFlake {} {
    nix = { stdenv, ... }: rec {
      pname = "nix";
      version = "2.4pre20200622_334e26b";

      outputs = [ "out" "dev" "man" "doc" ];

      depsBuildHost = [
        "pkgconfig"
        "autoreconfHook"
        "autoconf-archive"
        "bison"
        "flex"
        "libxml2"
        "libxslt"
        "docbook5"
        "docbook_xsl_ns"
        "jq"
      ];
      depsHostTarget = [
        "curl"
        "openssl"
        "sqlite"
        "xz"
        "bzip2"
        "nlohmann_json"
        "brotli"
        "boost"
        "editline"
        "libsodium"
        "libarchive"
        "gtest"
      ] ++ stdenv.lib.optional stdenv.hostPlatform.isLinux "libseccomp";
      depsHostTargetPropagated = [ "boehmgc" ];

      src = self;

      configureFlags = [
        "--with-store-dir=/nix/store"
        "--localstatedir=/nix/"
        "--sysconfdir=/etc"
        "--disable-init-state"
        "--enable-gc"
        "--with-system=${stdenv.hostPlatform.system}"
      ];

      makeFlags = [ "profiledir=${placeholder "out"}/etc/profile.d" ];

      installFlags = [ "sysconfdir=${placeholder "out"}/etc" ];
    };
  };
}

[garbas]

@mkenigs As already @matthewbauer showed you can use helper functions within flake.nix. There is also https://github.com/numtide/flake-utils which tries to simplify configuring some of the items in flake.nix.

As discussed with @edolstra over a coffee, right now flake.nix is quite verbose and this is with a reason that we can build more user friendly abstractions over it. It might even make sense to include flake-utils (or something similar) and ship it with Nix at some point, but only time will tell.

I'm going to close this issue because the ideas can already be explored and I expect a subsequent issue will be opened with a more direct proposal how to improve usability of flake.nix.