support nix environments

[jdx]

we should have similar support to direnv's use nix feature. Perhaps something like this:

[env]
"@nix" = "default.nix"
"@nix:flake" = "flake.nix"

I noticed https://github.com/nix-community/nix-direnv but only briefly glanced at it. Might be worth seeing if I can port similar performance considerations.

[jdx]

this is definitely something I could use a hand with if someone wanted to contribute. I don't use nix or understand it that well. I can help out with the rtx side of this if someone else could basically just make a script that exported the proper env vars (as of right now it isn't possible to do the @nix thing in the [env] section but that's something I can pretty easily add). You could come chat with me in the discord if you want to go over it with me.

[MatrixManAtYrService]

Some doubt about how direnv does it

I've spent a while using nix devshells (as defined in a flake.nix and activated via direnv's use flake). I've been doing this to encapsulate my entire projects' dev tooling. Ever since I encountered this bit of wisdom I've been questioning whether its a good idea.

The suggestion is that devshells are for debugging nix derivations, not necessarily for every little thing related to that project. So I think it would be nice if a flake could define a devshell separately from a list of packages mise would provide. This is a departure from direnv which just wants a shell to use.

How I'd do it

I made a gist with an example of how this might look: https://gist.github.com/MatrixManAtYrService/2e661cc9b7205e05da92de1778545a53 In this contrived example, pokemonsay and bb are tools that mise might be configured to provide, and cowsay belongs in exclusively in nix shell for use debugging strictly nix stuff.

Notice that the devshell PATH includes stuff like gnused, which came along for the ride when we asked for cowsay. By contrast, the references listed under the mise package only include the two that we asked for. I think that including less via mise means fewer opportunities for surprise interactions between one package's build inputs and another package's behavior at runtime. Problems like this are uncommon, but I find them hard to debug.

So I proposing that in addition to letting users indicate a devshell to mise, we also teach mise to accept packages from a flake definition, this way developers can package tools for their own consumption in a way that mirrors how they might package them for their user's consumption.

---

Lets do it?

I'm not a rust dev, and I just ran across mise today, but I'll help with this if I can because I'd probably use it often. The gist ☝️ should give you a feel for how we might go about populating the nix store and getting the paths we need.

To ensure my ideas here aren't totally crazy, I've consulted the nix forum for a sanity check on this idea. So far, signs point to not crazy.

[MatrixManAtYrService]

• I need to study https://github.com/chadac/mise-nix to see if it scratches this itch already.

• After a conversation in discord, I got this feedback:

I feel like what would be challenging is mise needs to know what the bin names of all the tools are

I feel like https://nix.dev/manual/nix/2.18/command-ref/new-cli/nix3-run.html#apps could be used to solve this, but I need to spend some time tinkering before I have a proposal for precisely how.

• Another comment from discord:

I think @MatrixManAtYrService's approach is more similar to integrating with something like poetry

I'll also take a look at how mise integrates with poetry. I've got homework to do. I'll be back....

[fricklerhandwerk]

[env]
"@nix" = "default.nix"
"@nix:flake" = "flake.nix"

A robust interface would at least allow specifying which attribute you want from the Nix expression, and what to do with that. Is it supposed to merely be a tree with stuff in it (such as the output of buildEnv) or something that produces a particular executable that will be run, such as a shell invocation?

[joshbode]

This isn't particularly robust, but I'm using this solution at the moment to automatically activate the flake dev-env, including some logic to prevent repeatedly regenerating the dev-env which can be quite expensive.

• .mise.toml:

  [env]
  _.source = "activate.sh"

• activate.sh:

  #! /usr/bin/env bash
  
  set -euo pipefail
  
  # find this script
  BASE_DIR=${BASH_SOURCE[0]%/*}
  if [[ ${BASE_DIR} == "." ]]; then
    BASE_DIR=${PWD}
  elif [[ ${BASE_DIR} == "${BASE_DIR#/}" ]]; then
    BASE_DIR=${PWD}/${BASE_DIR}
  fi
  
  FILES=(
    "${BASE_DIR}/flake.nix"
    "${BASE_DIR}/flake.lock"
  )
  
  read -r _ HASH < <(cat "${FILES[@]}" | openssl sha256)
  
  # check hash to prevent unnecessarily re-sourcing environment
  HASH_FILE=${TMPDIR%/}/mise-nix-${HASH}
  
  if [[ ! -e ${HASH_FILE} ]]; then
    # shellcheck source=/dev/null
    nix print-dev-env > "${HASH_FILE}"
  fi
  
  # shellcheck source=/dev/null
  source "${HASH_FILE}"
  
  export _NIX_MISE_PATH=${PATH}

The only issue for me is that it doesn't appear to be possible to set PATH from the sourced script, so I have set a precmd hook (I use zsh) to replace PATH with the exported _NIX_MISE_PATH, but it's a bit of a hack :|

function _fix_nix_mise {
  if [[ -v IN_NIX_SHELL ]]; then
    if [[ -v _NIX_MISE_PATH ]]; then
      _NIX_MISE_OLD_PATH=${PATH}
      PATH=${_NIX_MISE_PATH}
      unset _NIX_MISE_PATH
    fi
  else
    if [[ -v _NIX_MISE_OLD_PATH ]]; then
      PATH=${_NIX_MISE_OLD_PATH}
      unset _NIX_MISE_OLD_PATH
    fi
  fi
}
add-zsh-hook precmd _fix_nix_mise

[jdx]

I think ideally this logic would live in a plugin. We don't currently have support for plugins to create env._ modules but that has been my plan for a while.

There's some prior art we could leverage with the internal MISE_ADD_PATH env var:

mise/src/toolset/mod.rs

Lines 447 to 451 in 0f290f0

	let add_paths = entries
	.iter()
	.filter(|(k, _)| k == "MISE_ADD_PATH" || k == "RTX_ADD_PATH")
	.map(|(_, v)| v)
	.join(":");

Presumably we could use this inside of _.source (it doesn't use this currently but I don't think it would be hard to add) and inside of env plugins like an _.nix.

[joshbode]

My attempt at creating a plugin here:
https://github.com/joshbode/mise-nix

which works with a .mise.toml like:

[tools]
nix = { version = "latest" }

I wasn't sure what to do about "installing" nix, since it should already be present on the system (this is really outside of the scope of the plugin, in my opinion) so I'm using a placeholder version (_) and dummy list-all and install scripts as the version key is mandatory, and it seems like there needs to be a tool install for a tool to be activated by mise, but maybe there's a way around that.

Also, I tried to get the cache-key to depend upon the hashes of flake.nix and flake.lock but if I change either file the environment isn't being regenerated unless I cd out of the project then return, but perhaps I'm doing that wrong or the caching doesn't work that way.

[fricklerhandwerk]

@joshbode Nix maintainer here. What exactly is version supposed to express? As mentioned in #1302 (comment), a Nix expression will give you a dictionary you need to pick a key from, and you should know what sort of data you expect in there.

[jdx]

I wasn't sure what to do about "installing" nix, since it should already be present on the system (this is really outside of the scope of the plugin, in my opinion) so I'm using a placeholder version (_) and dummy list-all and install scripts as the version key is mandatory, and it seems like there needs to be a tool install for a tool to be activated by mise, but maybe there's a way around that.

Yeah this is the workaround you need since we don't have support for non-tool plugins like this. I don't think that would be hard to add though.

Also, I tried to get the cache-key to depend upon the hashes of flake.nix and flake.lock but if I change either file the environment isn't being regenerated unless I cd out of the project then return, but perhaps I'm doing that wrong or the caching doesn't work that way.

I think there is an internal config for plugins somewhere that can disable caching entirely if that would be better. Though I agree it should be using the timestamp of those files.

[joshbode]

Nix maintainer here. What exactly is version supposed to express? As mentioned in #1302 (comment), a Nix expression will give you a dictionary you need to pick a key from, and you should know what sort of data you expect in there.

@fricklerhandwerk it's purely a placeholder to get around the (current) requirement in Mise that tools plugins install a version of something, so the version key is mandatory. In the case of Nix this is unreasonable, since nix is going to need to be already on the user's system and installing Nix is very likely outside the scope of a Mise plugin.

As pointed out, this would probably be better as a plugin targeting env (which isn't currently supported) rather than a tools plugin like I've made.

Either way, we could definitely add an option to target a specific flake output attribute (rather than defaulting to devShells)

[jdx]

I've done some work to get env-only plugins: #2832

[joshbode]

It could do with some polish, but I've got a version of mise-nix using the new Lua/VFox-based env-only plugin, which is mostly doing what I expect. The experience is focussed on using flakes, but it should be possible to hook into a plain nix file with a basic flake wrapper.

Basically, a .mise.toml in the same directory as flake.nix should work:

[env]
_.nix = {}

This isn't in the plugin registry yet, so to try it out use:

$ mise plugins install https://github.com/joshbode/mise-nix.git

See https://github.com/joshbode/mise-nix/tree/main/test for a basic example.

If anyone has any feedback, please let me know!

Note: there's a slightly annoying log message printed before each prompt line which should go away in a new release soon (see: jdx/vfox.rs/pull/50)

[jdx]

[env]
_.nix = {  }

that's a little weird but I know why it's like that, maybe we could make this work instead?

[env]
_.nix = true

[joshbode]

Good idea, @jdx - it works either way now :)

[jdx]

Let me know if you have any feedback on the design here btw. I chose to do this via vfox instead of asdf since that allows windows support (of course, not useful for nix). I haven't tested but I also suspect lua is probably a lot faster than bash too.

[joshbode]

It's been pretty good! I think you're right about the performance - it seems faster.

I chose to do this via vfox instead of asdf since that allows windows support (of course, not useful for nix)
Oooh - that's handy! Yes - what I've made here definitely won't work on Windows (apart from nix) and it'll only work for probably zsh, bash (and maybe yash - I might be able to swing it for dash if I use POSIX tests, but I don't think mise loads there)

A few of comments:

• Lua has no built-in higher-level file/path operations (e.g. checking if files exist, forming paths, hashing files, etc) so I had to implement those, and I'm sure my implementations could be more robust. Potentially we might want to standardise these into a library (part of vfox.rs perhaps?) or find a good library to integrate - I'll bet there's an issue already in vfox for this - the included json and strings modules are useful, though :)
• I'm not sure if I'm abusing something, but it felt a little hacky to be using print to establish function definitions and non-exported variables (simply by printing their definition using POSIX shell syntax) - it's easy to mess up the quoting on this, and with the ability to return exports it might be nice to have the ability to optionally return functions and variables, also - then we could have a standard, managed cleanup of these (like with restoring exports) when the user navigates out of the project (I'm managing this myself with a hook, currently)
• Similarly, logging info to the user needs to be handled carefully, also, and I'm using carefully (enough?) quoted printf expressions to achieve this, so something standard might be useful (see utils.log).
• I'm performing caching of the generated environment (variables, functions) in TMPDIR, since nix print-env is too slow to invoke every time - something standard for this might help, also - perhaps something that looks at the hash of a collection of files to determine whether the cache should be updated - I think there is something like this in the existing mise.plugin.toml but I couldn't ever get it to work (e.g. https://github.com/joshbode/mise-nix/blob/bfca951d1a29fc424e2a436eae7da02833bc246c/mise.plugin.toml).
• I'm not using the path hook, since it seems like you can just do it from the env hook with a PATH key and that way I can do it all in one go :)
• Possibly something in the ctx for the MISE_PROJECT_ROOT (not available, it seems), and even the whole family of MISE_* variables (e.g. MISE_SHELL which I used) might be useful, too.

Overall, it's been nice, though - I think this is a good move.

If you get a chance to bump the vfox.rs dependency that would be ace, and we won't get a debug message on every new prompt line :) it looks like you did already - it feels like magic now that it's silent ;)

[joshbode]

I've removed the loading of functions and non-exported variables - something changed in the latest version of mise (perhaps the zsh performance-related changes) and the dubious method I was using to load/unload them broke (the emitted printed variables and functions were interleaved with the exports when I looked at the mise hook-env output).

I don't think they're of much use anyway (mostly nix-internal build-stage related variables and helper-functions) and if they're really needed then nix develop is still there for the full environment.