JIT cache implementation

[nrdxp]

Partially fixes #13.

The initial implementation allows a user to optionally pull the target derivation from the configured Nix cache. It's availability can be determined by using the cached output from the discover action.

This includes a refactoring of secrets into their own action so it can be used to setup secrets for both discovery and for the runners. All secret related inputs to the run action have been moved into this new action.

For projects that produce a large /nix/store in the discovery phase, this should drastically increase task startup time. In order to improve total action runtime further, we would need a more granular needs that can skip waiting on the post step for the discover action.

We should probably request this upstream, but I am not sure where would be appropriate. actions/toolkit perhaps?

[nrdxp]

So after getting this to "work" and testing it out, it is going to be even more expensive during the upload phase because querying the cache for existing paths is massively inefficient en masse currently: NixOS/nix#7221 (comment)

We could set a massive narinfo-cache-positive-ttl so that the query cache doesn't ever expire, but I'm not sure if that might cause other issues 🤔

Plus, even after waiting a full hour in the CI runner, the task cache never even finished querying paths, so I'm not sure if I'll ever be able to poppulate the query cache in the first place.

Update
Some basic math that I'm sure isn't strictly accurate but perhaps a rough estimate. I uploaded all the derivations from my local machine, there were about 30,000 individual uploads, but the part I want to focus on is the query time. After I ran the upload I did another cycle with all the ttl's set to 0 to see how long just a straight query would take. On my 12 core cpu it took 8m23s.

The default GH Linux runner has 2 cores, so some rough math would have us waiting aproximately 51 minutes just to query that the derivations already exist in the cache 🤦

[nrdxp]

Moving to draft unless/until we find a better solution to this. Our hopes for efficient persistent state in the case of heavy evaluations will have to rest on #14 for now.

This did give me an idea for a new project though: maybe at some point in the future we could write some kind of frontend that can index a Nix cache and provide cheaper query API. In order for it to be really useful though it would also need to know how to write to the cache, so you'd probably have to teach the nix cli how to talk to this frontend if it is available, rather than manually querying each path individually.

In any case, just a thought I wanted to get down in case we ever have time to work it out further.

[blaggacao]

For context of future readers.

This has progressed via #14 into:

ssh "$DISCOVERY_SSH" -- \ 
  'nix-store --export \ 
    $(nix-store --query --requisites \ 
      ${{ fromJSON(inputs.json).targetDrv }} \
      ${{ fromJSON(inputs.json).actionDrv }} \
    ) \ 
  | zstd' \ 
| unzstd \ 
| nix-store --import &>/dev/null