Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

$PATH is broken for fish shell #122

Open
grossbart opened this issue Jan 21, 2019 · 60 comments
Open

$PATH is broken for fish shell #122

grossbart opened this issue Jan 21, 2019 · 60 comments

Comments

@grossbart
Copy link

I updated nix-darwin today and had to rollback to a version from October because my $PATH was broken (using fish shell).

Before (correct):

/Users/gape/.nix-profile/bin
/run/current-system/sw/bin
/nix/var/nix/profiles/default/bin
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

After (broken):

/Users/gape/.local/bin
/Users/gape/.local/bin
/Users/gape/.nix-profile/bin
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin
/Users/gape/.nix-profile/bin
/run/current-system/sw/bin
/nix/var/nix/profiles/default/bin

The important part being that /run/currenty-system/sw/bin comes too late and the Nix provided binaries are not picked up correctly, resolving to /usr/bin/git for example. I'm also not sure where all this duplication is coming from …

I tried to get to the source of this error but was not able to figure it out exactly. What worked was to switch to an older generation and then to the newest generation again: in this case, the $PATH was set correctly. But when I exit fish shell and enter it again, the $PATH is broken.

It could be that this problem was introduced in 676ef10#diff-02a3bd02a5cdf5583b2e516a0e92d58a because the version of nix-darwin that worked for me was from 2018-10-17 and this is the major change that happened to the environment a few days later. But then maybe not because no one else has this problem?

@jhenahan
Copy link

Also hitting this after updating from a version from last week. Haven't narrowed down the issue, yet, but it seems like environment.systemPath isn't being picked up by fish after export.

@LnL7
Copy link
Owner

LnL7 commented Jan 22, 2019

The same thing happens for all shells so I'm not sure what the issue is.

@grossbart
Copy link
Author

I spent some time looking for this bug and wanted to post a small update, even though I have not found the source of the problem nor a solution …

I went through all the commits since October and could not find anything that looked particularly problematic with regards to this problem. The one commit that caught my eye (and that I mentioned in the initial post), introduced this:

# Prevent this file from being sourced by child shells.
export __NIX_DARWIN_SET_ENVIRONMENT_DONE=1

This seems suspect to me because it introduces a global env variable that might be unpredictable. But then I went into the generated nixos-env-preinit.fish file and echoed the PATH. I could not find a problem there, it was all working correctly.

I also changed the tests to test for the sequence of the Nix paths and then the system paths in order and that was also correct.

But whenever I start a shell in the newest nix-darwin, my PATH is wrong as described above. So there has to be some other hook somewhere that works on this. I will keep looking …

A last thing: if I would find a place where I think it goes wrong, how would I test this on my system? Would I have to build the installer and run that from my local repo?

@yurrriq
Copy link
Contributor

yurrriq commented Jan 28, 2019

For the record, I'm having similar issues, it looks like.

@LnL7
Copy link
Owner

LnL7 commented Jan 28, 2019

I'm not a fish user so I don't know what to look for but I'd take a look at NixOS/nixpkgs#45784.
Also verifying the following might help to narrow down the problem:

  1. Is this reproducible with nixpkgs-18.09-darwin or only nixpkgs-unstable/master?
  2. Does this also happen on nixos-unstable, could just be hidden because /usr is almost empty there.

@yurrriq
Copy link
Contributor

yurrriq commented Jan 28, 2019

Here are my pinned nixpkgs and nix-darwin:

{
  "owner": "NixOS",
  "repo": "nixpkgs-channels",
  "branch": "nixpkgs-18.09-darwin",
  "rev": "0b471f71fada5f1d9bb31037a5dfb1faa83134ba",
  "sha256": "148vh3602ckm1vbqgs07fxwpdla62h37wspgy0bkcycqdavh7ra5"
}
{
  "owner": "LnL7",
  "repo": "nix-darwin",
  "branch": "master",
  "rev": "629fa534988b7a402f4278c9445f0e20e5f03973",
  "sha256": "033xwz0gaqc89khbs1l3lxii5v54sqqyzag55xjyqrmaz7bbancy"
}

and the resulting $PATH:

$ for p in $PATH; echo $p; end
/Users/me/.rvm/gems/jruby-9.2.5.0/bin
/Users/me/.rvm/gems/jruby-9.2.5.0@global/bin
/Users/me/.rvm/rubies/jruby-9.2.5.0/bin
/Users/me/.rvm/bin
/Users/me/bin
/bin
/Users/me/.nix-profile/bin
/run/current-system/sw/bin
/nix/var/nix/profiles/default/bin
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

For me, it looks like it's just /bin that's too early, and to be honest, I'm not sure that wasn't the case prior to October.

@thefloweringash
Copy link
Contributor

thefloweringash commented Feb 9, 2019

I had a look at this, and I think the problem is that Fish on nix-darwin runs its initializers in an incompatible order.

On macOS, there's a path_helper system that allows adding to paths via /etc/paths.d/. For example, /etc/paths.d/40-XQuartz includes the line /opt/X11/bin. Part of its behaviour, is that anything it manages is placed before the existing values of PATH. From my reading of fish's startup, the fish implementation also has the same behaviour.

$ PATH=/bin:/sentinel:/sbin /usr/libexec/path_helper -s
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/sentinel"; export PATH;

Logging the startup of a fish shell (without $__NIX_DARWIN_SET_ENVIRONMENT_DONE, to simulate a clean environment), shows that the nix-darwin path configuration is done before path helper.

$ fish --interactive --login -d3 -c 'echo $PATH' 2>&1  | cat -n | grep -e 'path_helper' -e 'fenv source'
   645  <3> fish: Created job 4 from command 'fenv source /nix/store/li0kdlly65x6a3n5h7jwn2l71v6f7zkk-set-environment' with pgrp -2
   691  <3> fish: Created job 3 from command 'command -sq /usr/libexec/path_helper' with pgrp -2

Nixpkgs does some additional configuration of fish to setup the environment very early, even before any of fish's main configuration is applied. Nix-darwin uses this same setup to inject the PATH variable. The eventual result is that the PATH is applied, then fish re-orders it in it's path_helper implementation.

A possible solution would be to stash the initial value of PATH in the pre-init, and restore it at the top of the regular init. I'll work on a PR.

While we're here: child shells

The behaviour around __NIX_DARWIN_SET_ENVIRONMENT_DONE should only apply to child shells, and is necessary for things like nix run to work. Prior to this change, every shell would explicitly reset PATH to the global configured value, and clobber whatever setup had been done by nix run, or user config in the parent shell, etc.

Fish also makes the questionable choice of running their path_helper on all shells. This is a departure from macOS, which only runs in login shells via /etc/profile. Even if nix-darwin does the right thing, this will cause all child shells to have their path's shuffled, breaking things like nix run[1].


[1] Presently nix run defaults to bash, so this would only be a problem when running nix run -c fish.

@thefloweringash
Copy link
Contributor

I think the required configuration point is missing in fish. I've filed an issue upstream at fish-shell/fish-shell#5641.

@grossbart
Copy link
Author

Thanks for the research, @thefloweringash. This rabbit hole gets deeper and deeper 😓 …

@yurrriq
Copy link
Contributor

yurrriq commented Apr 9, 2019

I ended up doing this:

for p in /run/current-system/sw/bin ~/bin
    if not contains $p $fish_user_paths
        set -g fish_user_paths $p $fish_user_paths
    end
end

... in my programs.fish.shellInit, as per @mqudsi's suggestion, and now my stuff works well enough.

🤷‍♂️ YMMV

fish, version 3.0.1

@grossbart
Copy link
Author

Thanks for the pointer, @yurrriq, this works for me, too. I'm on fish 3.0.0 and without this I still had the problem. It's hackish, but it works … This is what I added to my .nixpkgs/darwin-configuration.nix:

  programs.fish.enable = true;
  programs.fish.shellInit = ''
    for p in /run/current-system/sw/bin
      if not contains $p $fish_user_paths
        set -g fish_user_paths $p $fish_user_paths
      end
    end
  '';

@LnL7
Copy link
Owner

LnL7 commented Apr 15, 2019

If that also works for the version of fish in 18.09 something similar could be added to the module by default.

@yurrriq
Copy link
Contributor

yurrriq commented Apr 16, 2019

I'm using 18.09 (NixOS/nixpkgs@46d3867) on NixOS and it seems to work for fish 2.7.1 too.

@what-the-functor
Copy link

I also ran into this. Fish version 3.0.2. @yurrriq's fix works for me, as well.

@teehemkay
Copy link

teehemkay commented Aug 2, 2019

I have the following function definition in programs.fish.shellInit

      function __nix_darwin_fish_macos_fix_path -d "reorder path prioritizing darwin-nix paths"
        # Fish initialization for login shells rebuilds $PATH by emulating calling /usr/libexec/path_helper
        # which results in nix-darwin additional paths being appended (instead of prependd) to $PATH
        # (see https://github.com/fish-shell/fish-shell/blob/90547a861a13806cdfcf479e279527b2cb18c922/share/config.fish#L171)
        # As a workaround we re-source the nix-darwin environment again if necessary
        if test $PATH[1] = '/usr/local/bin'
          set fish_function_path ${pkgs.fish-foreign-env}/share/fish-foreign-env/functions $fish_function_path
          # source the NixOS environment config again
          fenv source ${config.system.build.setEnvironment}
          set -e fish_function_path[1]
        end
      end
      '';

then I just invoke __nix_darwin_fish_macos_fix_path at the top of programs.fish.interactiveShellInit.

The first test in the function may need to be adapted though.

HTH

@anka-213
Copy link

Here's a couple of suggestions for solutions from faho on fish-shell/fish-shell#7142 (comment)

  • Remove the offending paths from /etc/paths or $fish_user_paths - this is probably not viable for nix
  • Add the nix paths to /etc/paths - this might be viable for nix-darwin
  • Use a configuration snippet. Call it "00-nix.fish", put the path manipulation there. This is clean, simple and does what you want. It can be kept out of sight of users by putting it in the system conf.d
  • Use $fish_user_paths - this is more visible to the user and can cause issues should they erase the variable
  • Use /etc/fish/config.fish - this is semi-visible to the user, and only works after the snippets, so it's probably not viable
  • Patch share/config.fish to add in $PATH changes later - note that we don't want __fish_build_paths later because that's needed to find our functions, of which we execute a few in share/config.fish

I'm not sure which of these (if any) is most viable.

@happysalada
Copy link
Contributor

For me a simple fix was setting
environment.systemPath = [ /run/current-system/sw/bin ];
working with all shells

@webframp
Copy link

webframp commented Oct 9, 2020

Seeing this same problem with zsh

@LnL7 LnL7 changed the title $PATH is broken for fish shell (and I think for bash, too) $PATH is broken for fish shell Dec 19, 2020
@lilyball
Copy link
Contributor

We should be able to stuff the Nix stuff into fish_user_paths to get around this. My existing nixpkgs fish configuration looks like

{
  fish = super.fish.override {
    fishEnvPreInit = sourceBash: ''
      set -l oldPath $PATH
    '' + sourceBash profilePath + ''
      for elt in $PATH
        if not contains -- $elt $oldPath
          set -ag fish_user_paths $elt
        end
      end
      set PATH $oldPath
    '';
  };
}

It records the $PATH before sourcing the Nix environment, then walks the new $PATH and copies anything new into fish_user_paths, then restores the old $PATH. This code runs in the equivalent spot as fish/nixos-env-preinit.fish. So we could do the same thing here. The downside of course is the user then has to avoid overwriting fish_user_paths, but there's no other obvious workaround as $__fish_data_dir/config.fish slaps the macOS default paths on the front, calls the function to move fish_user_paths to the front, and then sources the conf.d/* files, and there's no opportunity to run code after the PATH modification and before ~/.config/fish/conf.d/*` is sourced.

Perhaps the best action here is to add it to fish_user_paths, but to not remove it from $PATH. It looks like any entry in fish_user_paths that already existed in $PATH is still moved to the front of $PATH but is not marked as something to be deleted when fish_user_paths is reset. This way users who unconditionally overwrite fish_user_paths will retain their Nix environment, but we'll have successfully protected it from the /usr/libexec/path_helper code.

And by doing it this way, the $PATH will be correct during the sourcing of the conf.d files.

@lilyball
Copy link
Contributor

I just tested this by hacking my config file with mkBefore and mkAfter and it seems to work, although it turns out the nix-darwin environment includes /usr/bin:/bin:… so all of those end up in $fish_user_paths, which is a bit weird. I suppose I could explicitly list those to be skipped.

@lilyball
Copy link
Contributor

Here's my hack:

{ config, lib, ... }:

with lib;

let
  cfg = config.programs.fish;
in
{
  config = mkIf cfg.enable {
    environment.etc."fish/nixos-env-preinit.fish".text = mkMerge [
      (mkBefore ''
        set -l oldPath $PATH
      '')
      (mkAfter ''
        for elt in $PATH
          if not contains -- $elt $oldPath /usr/local/bin /usr/bin /bin /usr/sbin /sbin
            set -ag fish_user_paths $elt
          end
        end
        set -el oldPath
      '')
    ];
  };
}

My resulting $PATH looks correct, and clearing $fish_user_paths afterwards doesn't break it.

bcc32 added a commit to bcc32/prezto that referenced this issue May 27, 2021
This is a workaround for macOS's path_helper stuff in /etc/zprofile:

- NixOS/nix#4169
- LnL7/nix-darwin#122
@winterqt
Copy link

Here's another hack that increases compatibility a bit in certain cases (like login shells):

  environment.etc."paths" = {
    text = concatStringsSep "\n" ([ "/Users/winter/.nix-profile/bin" ] ++ (remove "$HOME/.nix-profile/bin" (splitString ":" config.environment.systemPath)));
    knownSha256Hashes = [
      "cdfc5a48233b2f44bc18da0cf5e26df47e9424820793d53886aa175dfbca7896"
    ];
  };

Obviously only works for single user systems, replace /Users/winter with your home directory.

@pingiun
Copy link

pingiun commented Nov 17, 2021

In https://github.com/fish-shell/fish-shell/blob/e066715127fc0a515e7a8c66a2b7b21531bd500e/share/config.fish#L199 I think this change is needed:

             # Merge in any existing path elements
             for existing_entry in $$argv[1]
                 if not contains -- $existing_entry $result
-                    set -a result $existing_entry
+                    set -p result $existing_entry
                 end
             end

When I hacked this into my file in /nix/store, the resulting path was

; echo $PATH
/nix/var/nix/profiles/default/bin /run/current-system/sw/bin /Users/jelle/.nix-profile/bin /usr/local/bin /usr/bin /bin /usr/sbin /sbin /Library/Apple/usr/bin /Applications/VMware Fusion Tech Preview.app/Contents/Public /usr/local/MacGPG2/bin

which seems like it's completely correct.

@lilyball
Copy link
Contributor

@pingiun That reverses the order of the existing entries. Also this code here is mimicking the behavior of /usr/libexec/path_helper. I'm not actually sure why path_helper prepends the paths though, especially since its manpage says it's supposed to append them. Perhaps because appending then runs into ordering issues, where the paths it sets are supposed to have a particular order but any that already exist in PATH would screw that up.

This issue affects nested bash login shells too (since it uses path_helper directly), so it's not just Fish.

The discussion in fish-shell/fish-shell#7142 (thank you @anka-213) goes into detail about the problem here. I just commented as I think a real solution does unfortunately require modifying share/config.fish, though I'm not optimistic about getting any upstream changes. And sadly we cannot simply add a file to /etc/paths.d/ as any paths installed that way come after the system paths in /etc/paths.

Until such time as we get an upstream change in Fish, I think our only real solution is to patch share/config.fish to insert the hook we need. The hook can be a simple emit __fish_config_macos_set_env_post immediately after the __fish_macos_set_env call, and nix-darwin can define a function to look for that in nixos-env-preinit.fish and use it to fix up the path. This does mean changing the Fish derivation though, which means recompiling, so this patch would have to go into nixpkgs.

@mamciek
Copy link

mamciek commented Nov 29, 2022

I had another reinstall, this time it included fresh OSX reinstall as well. I don't have this problem now.

@layday
Copy link

layday commented Nov 29, 2022

Until the time you update Nix and you are downgraded to 2.11.0 ;)

@mathieupost
Copy link

Until the time you update Nix and you are downgraded to 2.11.0 ;)

❯ nix --version
nix (Nix) 2.11.0

Still works, as the fix is in the install script ;)

@layday
Copy link

layday commented Dec 1, 2022

Hmm, perhaps it's something that Home Manager is doing.

@layday
Copy link

layday commented Dec 1, 2022

Right, the installer will update fish config files in place. That is, this'll only work if you have previously installed fish and did not wipe its config files. Considering that macOS doesn't come with fish, and if you intend to use nix as a replacement for homebrew/macports, i.e. if you are gonna install fish with nix, then you need to be on nix 2.11.1 to be able to source the new nix profile scripts for fish, which are generated when building nix. If you've upgraded (downgraded) nix after a fresh install, you can switch back to the 2.11.1 generation with nix-env.

@tacomilkshake
Copy link

tacomilkshake commented Dec 10, 2022

If you have nix 2.11.1 or later installed in your user profile (via nix-env or nix profile), nix will only add paths for binaries installed in your user profile and the root user's user profile.

This won't add paths for system installed packages--which is absolutely needed. Unfortunately, you need to add /run/current-system/sw/bin/ after the paths added by nix 2.11.1. This means you have to provide an override so that all three nix binary paths are in the correct order (user, root, system), overriding any benefit from the 2.11.1 fixes.

So, nix 2.11.1 is still broken with fish on Darwin.

To patch this, I added this to the top of my ~/.config/fish/config.fish:

### Add nix binary paths to the PATH
# Perhaps someday will be fixed in nix or nix-darwin itself
if test (uname) = Darwin
    fish_add_path --prepend --global "$HOME/.nix-profile/bin" /nix/var/nix/profiles/default/bin /run/current-system/sw/bin
end

I included the test for Darwin as this only occurs on my darwin systems. nixos + fish paths are set just fine.

@arichtman
Copy link

I've encountered this too - just on the stock zsh

nix files

arichtman@macbookpro ~ % nix --version
nix (Nix) 2.13.2
arichtman@macbookpro ~ % uname -a 
Darwin macbookpro 21.6.0 Darwin Kernel Version 21.6.0: Mon Dec 19 20:44:01 PST 2022; root:xnu-8020.240.18~2/RELEASE_X86_64 x86_64

@mathieupost
Copy link

On my new MacBook it unfortunately doesn't work anymore, so I'm back to this workaround

@khaneliman
Copy link
Contributor

If you have nix 2.11.1 or later installed in your user profile (via nix-env or nix profile), nix will only add paths for binaries installed in your user profile and the root user's user profile.

This won't add paths for system installed packages--which is absolutely needed. Unfortunately, you need to add /run/current-system/sw/bin/ after the paths added by nix 2.11.1. This means you have to provide an override so that all three nix binary paths are in the correct order (user, root, system), overriding any benefit from the 2.11.1 fixes.

So, nix 2.11.1 is still broken with fish on Darwin.

To patch this, I added this to the top of my ~/.config/fish/config.fish:

### Add nix binary paths to the PATH
# Perhaps someday will be fixed in nix or nix-darwin itself
if test (uname) = Darwin
    fish_add_path --prepend --global "$HOME/.nix-profile/bin" /nix/var/nix/profiles/default/bin /run/current-system/sw/bin
end

I included the test for Darwin as this only occurs on my darwin systems. nixos + fish paths are set just fine.

Fresh install of nix and nix-darwin and this worked for fixing it for me. Thanks :)

@clo4
Copy link

clo4 commented Aug 1, 2023

This is a quick recap of this issue for people without much experience with Nix internals (like me!), people that don't want to read the thread, or people that just want a solid solution.

What's the problem?

The script nixos-env-preinit.fish is executed on shell startup. You can find this at /etc/fish/nixos-env-preinit.fish on your system. This file executes another script that sets up environment variables, including the $PATH. The path is set up correctly here!

However, this happens before fish runs its own config scripts. Only when Fish is set as your login shell, it reads from /etc/paths and /etc/paths.d/*, sets that as the path, and then appends the pre-existing $PATH. Here's the code that does this. This is normally what you want, but in this case, it ends up putting your Nix directories at the end.

What paths exactly need to be moved?

Before Fish clobbers the path variable, the important directories in it are the /bin subdirectories of the values of environment.profiles.
This is set, by default, by:

... but anything can add to this value. Using hardcoded paths is probably fine but isn't completely "correct" in the programmer-y ideology kind of way.

What's the fix?

The paths in environment.profiles need to be moved to the front of $PATH but behind $fish_user_paths, which is a magic fish variable that's always prepended to your path. (When you use the fish_add_path command, it prepends to $fish_user_paths universally by default. Universal means it modifies ~/.config/fish/fish_variables)

This module works correctly -- you can copy/paste it or save it as a file and add it to imports. I'm pretty new to Nix so this might not be super idiomatic, but this is about as correct as it can be I think.

{ lib, config, ... }:
{
  programs.fish.loginShellInit =
    let
      # This naive quoting is good enough in this case. There shouldn't be any
      # double quotes in the input string, and it needs to be double quoted in case
      # it contains a space (which is unlikely!)
      dquote = str: "\"" + str + "\"";

      makeBinPathList = map (path: path + "/bin");
    in ''
      fish_add_path --move --prepend --path ${lib.concatMapStringsSep " " dquote (makeBinPathList config.environment.profiles)}
      set fish_user_paths $fish_user_paths
    '';
}

You can probably get away without double quoting at all, it just makes me a little uncomfortable to think that a stray space could blow up my shell init 😅

The set fish_user_paths $fish_user_paths line is done for the side-effect of reordering $PATH, putting the user paths in the front.

(There are lots of valid ways of doing this -- another is to use string split " " $NIX_PROFILES)[-1..1]/bin, but this will always happen at runtime)

This is valid either in your home-manager config or nix-darwin config, both define programs.fish.loginShellInit, but as @yoav-lavi noted, you need to change config to osConfig when using it as a home-manager module.

A note about Fish's weird source order

Fish is kinda weird about what order to source things in.
Currently, as of fish 3.6, fish sources user config first (~/.config/fish/conf.d/), then system config (/etc/fish/conf.d/), then vendor config, then user config.fish, and only then the system config.fish. This means using programs.fish.loginShellInit will execute last. Everything that gets executed before that will have the wrong path, but your interactive shell will be correct.

This will be fixed in the future by sourcing config snippets (conf.d/*.fish) in order of name, regardless of the directory. So the most correct thing to do would be using something like

environment.etc."fish/conf.d/00-nix.fish".text =
  let
    # ...
  in ''
    if status is-login
      # config snippet...
    end
  '';

But again, this doesn't really matter that much right now, because files under /etc/fish will always be executed after your user config.


There's also the issue that fish doesn't technically source /etc/fish/..., it looks in $__fish_sysconf_dir which is set at compile time to a location in the nix store. The only reason /etc/fish/config.fish is executed is because of a shim). But nothing knows about /etc/fish/conf.d, so it's not used.

To fix this, maybe it's worth changing nixpkgs' fish preinit to add /etc/fish to __extra_confdir? I'm not sure what the ramifications would be for this ...

@yoav-lavi
Copy link

@clo4 note that if you plan on using this in a home-manager managed fish shell, you'll need to use osConfig.environment.profiles rather than config.environment.profiles (at least that's the only thing that ended up working for me)

@khaneliman
Copy link
Contributor

@clo4 note that if you plan on using this in a home-manager managed fish shell, you'll need to use osConfig.environment.profiles rather than config.environment.profiles (at least that's the only thing that ended up working for me)

This helped me when I transitioned to using home-manager configs for my shell on my macbook. Thanks.

dispanser added a commit to dispanser/dot-files that referenced this issue Sep 11, 2023
This helps placing the nix paths in front of the $PATH variable, so that
they are preferred over macos-provided or homebrew stuff.

See LnL7/nix-darwin#122 for a lengthy
discussion.
@britter
Copy link

britter commented Oct 27, 2023

Hey, I'm experiencing the problem with binaries installed via nix-darwin and home-manager not being present in PATH, but for me it looks like the root cause is different in my case.

Context

I'm migrating an existing homebrew setup over to a flake base nix-darwin setup with home-manager. I installed nix using the Determinate Systems installer. Right after the first nix run nix-darwin -- switch --flake . I noticed that darwin-rebuild was not in my PATH. I ignored that for the beginning and started migrating stuff from homebrew over to my new home-manager setup. After migrating my fish configuration and uninstalling the homebrew fish install, all the binaries installed via home-manager didn't work in fish anymore. In fact, there terminal app didn't work anymore because /opt/homebrew/bin/fish was removed.

Root cause

The root cause seems to be that I had configured chsh -s /opt/homebrew/bin/fish. That led to the "wrong" fish installation being started when I opened the terminal. When I start fish via /run/current-system/sw/bin/fish the path is correct. My expectation is, that nix-darwin and/or home-manager take care of configuring this for me if I set users.users.my-user.shell = pkgs.fish and programs.fish.enabled = true but it doesn't seem to be the case. Also the fish installed from nix is not present in /etc/shells and can therefore not be selected using chsh

Fix

  • Make sure the fish install from nix-darwin ends up in /etc/shells
  • chsh to the fish shell coming from nix-darwin

Workaround

  • Manually add /run/current-system/sw/bin/fish to /etc/shells
  • Run chsh -s /run/current-system/sw/bin/fish
  • Restart the terminal

PATH should be correct now.

@lilyball
Copy link
Contributor

@britter The problem is nix-darwin does not modify any user accounts unless they're listed in users.knownUsers, but that warns you not to put the admin user or other system users there because having a user listed there means nix-darwin is free to create/delete it or change whatever it wants. Also, skimming the implementation right now, I'm not sure it applies any changes to user options to an already-existing user, it looks like it just sets everything when it creates the user.

In my local setup I have my own activation script that explicitly checks for and updates my user shell because of this issue (and also this code actually sets the shell to a wrapper program that basically does /bin/wait4path /nix/store && exec $shell "$@", except it's a C program instead of a shell script wrapper to avoid issues with argv[0]).

@britter
Copy link

britter commented Oct 31, 2023

@lilyball can you point me to the documentation of users.knownUsers? I tried finding something about it but failed.

I think there are two things at play here:

  1. setting programs.fish.enable = true should add fish to etc/shells because otherwise it's unusable. On my system that didn't happen.
  2. When setting pkgs.fish as defaultShell for my user, I expect it to chsh -s to fish, so fish becomes my default shell.

This is what happens on NixOS, but it doesn't seem to work on my macOS machine. Curious to understand if I made a mistake or if I just have the wrong expectations.

@lilyball
Copy link
Contributor

lilyball commented Nov 1, 2023

@britter

users.knownUsers = mkOption {
type = types.listOf types.str;
default = [];
description = lib.mdDoc ''
List of users owned and managed by nix-darwin. Used to indicate
what users are safe to create/delete based on the configuration.
Don't add the admin user or other system users to this.
'';
};

system.activationScripts.users.text = mkIf (cfg.knownUsers != []) ''

If you look at the whole activation script here you can see that it calculates a list of "created" and "deleted" users by cross-referencing the users.knownUsers list with the users.users set. For every "created" user, if users.forceRecreate is set it first deletes the user (if it exists) and then recreates it, and if that's not set then it simply skips any user that already exists (meaning it won't apply any changes to user properties).

I feel like this whole module needs to be rewritten such that it will sync changes to known users.

For any user not listed in the known users list, the properties you can set effectively act as ways to notify nix-darwin as to what your current user config is, in case anything wants to reference that. For example, I use this to tell it where the home folder for my user is, so home-manager knows where to write its config to.

As for /etc/shells, as best I can figure, that file actually doesn't matter for most things on macOS anymore. I don't even bother updating that with my config anymore. The comment in the file as it ships with macOS says that Ftpd consults it, but I guess nothing else does.

@jjant
Copy link

jjant commented Dec 1, 2023

Can we merge @clo4's fix into nix-darwin, @LnL7?

@zhou13
Copy link

zhou13 commented May 13, 2024

For people having issues with zsh, the following seems working for me:

        programs.zsh = {
          enable = true;
          loginShellInit = ". /etc/zprofile\n";
        };

@brendanzab
Copy link

This is the solution I’m now using, which takes into account the new XDG locations, as described in the nix manual:

    # FIXME: This is needed to address bug where the $PATH is re-ordered by
    # the `path_helper` tool, prioritising Apple’s tools over the ones we’ve
    # installed with nix.
    #
    # This gist explains the issue in more detail: https://gist.github.com/Linerre/f11ad4a6a934dcf01ee8415c9457e7b2
    # There is also an issue open for nix-darwin: https://github.com/LnL7/nix-darwin/issues/122
    programs.fish.loginShellInit =
      let
        # We should probably use `config.environment.profiles`, as described in
        # https://github.com/LnL7/nix-darwin/issues/122#issuecomment-1659465635
        # but this takes into account the new XDG paths used when the nix
        # configuration has `use-xdg-base-directories` enabled. See:
        # https://github.com/LnL7/nix-darwin/issues/947 for more information.
        profiles = [
          "/etc/profiles/per-user/$USER" # Home manager packages
          "$HOME/.nix-profile"
          "(set -q XDG_STATE_HOME; and echo $XDG_STATE_HOME; or echo $HOME/.local/state)/nix/profile"
          "/run/current-system/sw"
          "/nix/var/nix/profiles/default"
        ];

        makeBinSearchPath =
          lib.concatMapStringsSep " " (path: "${path}/bin");
      in
      ''
        # Fix path that was re-ordered by Apple's path_helper
        fish_add_path --move --prepend --path ${makeBinSearchPath profiles}
        set fish_user_paths $fish_user_paths
      '';

petertriho added a commit to petertriho/nix-config that referenced this issue Aug 30, 2024
shanyouli added a commit to shanyouli/dotfiles that referenced this issue Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests