lib.fileset needs something like lib.cleanSource

[infinisil]

Issue description

lib.cleanSource filters out some of the most commonly unneeded files for you by default:

1. The .git directory/file, but also other version control systems like .svn or .hg
2. Editor backup/swap files like default.nix~
3. Generated files like .o and .so (this one seems a bit weird to me)
4. The result symlinks
5. Files with unknown type, they can't be imported into the store anyways

While it's possible to use

lib.fileset.fromSource (lib.cleanSource ./.)

this is clunky and makes the fileset library reliant on lib.sources. We should eventually be able to deprecate lib.sources at some point, but then we need a replacement.

One can also set up manual filters, but it's not pretty:

lib.fileset.difference ./. (lib.fileset.unions [
  ./.git
  (fileFilter (file:
    file.hasExt "o"
    || file.hasExt "so"
    || lib.hasSuffix "~"
    || file.type == "symlink" && lib.hasPrefix "result" file.name
    || file.type == "unknown"
  ) ./.)
])

Ideas

lib.fileset.clean

One simple idea is to have a lib.fileset.clean ./. that does kind of the same.

I don't like how the filtered out files are fairly arbitrary though. What if another editor comes along and uses the .blorp file extension to store its swap files, would we add it to this function and risk breaking users? The file set library should stay backwards compatible.

Filter library

So another idea would be to allow creating a sort of library of filters, something like

{
  fileset.garbageFileFilter = {
    c = file: file.hasExt "o";
    emacs = file: lib.hasSuffix "~" file.name;
    nix = file: lib.hasPrefix "result" file.name && file.type == "symlink";
  };
}

Then these could be combined in a fileFilter as necessary. Maybe a bit too manual though.

Versioned lib.fileset.clean

Another idea (which could also use the above) would be to create a versioned clean function like

{
  fileset.clean = {
    v1 = fileFilter (file: ...);
    v2 = fileFilter (file: ...);
  };
}

Where each single version stays backwards compatible, but new ones can always be introduced as necessary.

---

This issue is sponsored by Antithesis ✨