Add another branch with Haskell IDE Engine

[Skyfold]

I've been getting hie working locally to see how it works in practice. There are a few downsides to getting it setup: You need to either compile it yourself (2 hours plus) or set your nixpkgs to the same checkout as one of the pre-built ones that you can get on cachix (see this repo). I've been running into issues if I use a different checkout of nixpkgs even if I use the correct version of ghc and cabal. There is also a hack I need to write to get hie to find the hoogle database. Either way I would want to put the setup with hie in another branch so users can choose to just use ghcid (which always works).

Note: Setting up cachix is more difficult than I thought since we cannot just upload the shell derivation since it will be different for each user. Instead we would need to build the parts individually.

[utdemir]

I've been thinking about the same thing. I was thining about making it optional (cookiecutter is supposed to have conditionals, we can even ask them to choose between ghcid and hie like this), and probably would let them compile it themselves. In the longer term it'd make sense to submit a patch to nixpkgs so it'll be on cache.nixos.org.

I feel like cachix support would be tricky since it requires the user to be either a trusted Nix user or change the system configuration.

Also, I guess we have to wait until infinisil/all-hies#55 is fixed.

[Skyfold]

Making hie vs ghcid a choice in the cookiecutter template is better than having a separate branch. It looks like building hie with nix for 8.8.3 is not going to happen for awhile given Infinisil is trying to find a general fix for stack2nix not being maintained and alternatives like haskell.nix don't support enough versions of ghc.

I think we may have to create our own build (like this gist) and post the build on cachix ourselves so users don't have to build it themselves. We would only have to support the version of ghc that we pick for this repo or only the ones supported by haskell.nix.

Ideally we won't have to have different nixpkgs versions for the hie setup. In other words we try to pick a checkout of nixpkgs that works for development with and without hie.

Steps needed to get hie added to this template:

• build hie with haskell.nix and test locally
• see what versions of ghc work
• make sure checkout of nixpkgs has building haskell libraries for most cases (like hedgehog)
• push builds to cachix
• add cachix instructions to README
• setup template to ask "ghcid or hie"

Note: I agree cachix setup can be non-trivial and it would be much better to get hie on cache.nixos.org, but I'm not sure what else we can do in the meantime. We could setup our own cache with herculies-ci since I have heard hydra requires arcane knowledge. Its just more time consuming to setup and maintain. At the same time it would be nice to eventually automate some of what we are doing.

[utdemir]

We could setup our own cache with herculies-ci since I have heard hydra requires arcane knowledge. Its just more time consuming to setup and maintain. At the same time it would be nice to eventually automate some of what we are doing.

I already have CI setup (via buildkite) for this project, should be easy to introduce cachix there. I'll spend some time within this week on it.

[Skyfold]

I did some playing around with ghcide (hie is slow) and I have to say its a much better experience. The main downside is ghcide does not support Multi-Cradle correctly see issue #113. What this actually means is ghcide can only handle one set of dependencies for your entire project. This is because ghcide is essentially a wrapper around ghci (like ghcid) and can only load one target from your cabal file. For example, if you set your target to be your library then your test-suite won't build because its missing hedgehog. The workaround is to make the target your test-suite and have it contain all the dependencies of all targets. As long as you don't put them in an executable or the library of your project it shouldn't affect downstream projects. I think I can adjust the cabal file to do this for you, but I haven't worked it out yet.

Either way, I'll create a ghcide branch to show you would it would look like.

[Skyfold]

I created the ghcide branch with the extra hie.yaml that is necessary, but have not yet updated the readme. You will want to run cachix use ghcide-nix from https://github.com/cachix/ghcide-nix, so you won't have to build ghcide yourself.

There is a real annoyance I've run into, adding dependencies does not work like you would want to. If you update your cabal file while you have ghcide running, it will call cabal to pull in the new dependencies directly from hackage, not from your package set (It just calls cabal new-repl for the flags). What you want is to have all the packages in myHaskellPackages be available for if/when you add them to your cabal file. Unfortunately, this would mean we cannot use shellFor which uses your cabal file to just pull in the packages you need. Essentially, shellFor lets you download less to get a shell running, but at the cost of exiting your editor each time you add a new dependency. I'm not sure if there is a better way.

I might be able to setup something with sos in a separate terminal. I'll look into this tomorrow.

[Skyfold]

I spent some time trying out a few alternatives:

1. Getting your editor to run nix-shell --run "ghcide --lsp". Doesn't work since your editor tries talking to the nix-shell instead of ghcide (as far as I can tell).

2. Running nix-shell --run "ghcide --lsp" in a separate terminal and getting your editor to talk to it. Difficult to setup, but should be possible depending on what editor or editor plugin you are using.

3. Wrapping nix-shell --run "ghicde --lsp" in sos, requires you to edit watched files twice before it actually reloads. Could use another program to watch files, still suffers from getting it linked with your editor and losing completions while your environment is re-built.

In general any solution with nix-shell is less than ideal given how long it takes even when there are no changes. I wish lorri shell could do lorri shell --cached --run "ghcide --lsp".

Regardless, ghcide always reloads when your cabal file changes and that means you can lose completions until you restart ghcide with an updated environment.

I think the least painful setup is to have all of myHaskellPackages in your environment and just have hoogle load those from your cabal file. I'll see if I can put that together.

[Skyfold]

Well, I found one workaround, but it requires you to have a script in the repository (call it with ./ghcide-wrapped in your editor):

#! /usr/bin/env nix-shell
#! nix-shell --pure --run "ghcide --lsp"

You have your editor run this script instead of running ghcide directly. That means when you reload your server it sources the new environment. This does not fix ghcide restarting when your cabal file changes and nix-shell is slow to start, but it works better than before.

I tried adding this script to default.nix, but when I run the script from the nix store it tries to find a shell.nix in the nix store instead of in the directory the script was run.

[utdemir]

That is a cool trick!

I wonder if it'd work if wrote a shell script wrapper to ghcide and added it to our nix-shell. So when "ghcid" is invoked inside our nix-shell, it'll run nix-shell --pure --run "ghcide --lsp" instead.

[Skyfold]

Here is what I tried to set that up (though I called it ghcide-wrapped):

  mkScript = name: contents: let
      script = builtins.toFile name contents;
      makeExectuable = builtins.toFile "change-ownership" ''
          export PATH="$coreutils/bin"
          mkdir -p $out/bin
          cp ${script} $out/bin/$name
          chmod +x $out/bin/$name
        '';
    in
      derivation {
        inherit name;
        builder = "${pkgs.bash}/bin/bash";
        args = [ makeExectuable ];
        system = builtins.currentSystem;
        coreutils = pkgs.coreutils;
      };

(There might be a mkScript equivalent in nixpkgs, but I didn't find one)

Then add this to your buildInputs:

      (mkScript "ghcide-wrapped" ''
        #! /usr/bin/env nix-shell
        #! nix-shell --pure --run "ghcide --lsp" ./shell.nix
      '')

(This will add ghcide-wrapped to your path)

However, I kept getting error: getting status of '/nix/store/9cll74sf1wg90mm65jg4b64czdkz99as-ghcide-wrapped/bin/shell.nix': No such file or directory. Whats strange is when you run the script by itself, as in just as a regular file in your repository, it does not assume that the shell.nix file is in the same directory as the script.

[codygman]

Well, I found one workaround, but it requires you to have a script in the repository (call it with ./ghcide-wrapped in your editor):

#! /usr/bin/env nix-shell
#! nix-shell --pure --run "ghcide --lsp"

You have your editor run this script instead of running ghcide directly.

Doesn't direnv integration in editors solve this?

For instance emacs-direnv would essentially make ghcide do the same thing for lsp integration I believe.

[utdemir]

I think we may have to create our own build (like this gist) and post the build on cachix ourselves so users don't have to build it themselves.

I just fixed the CI, made it public, and configured it so that it uploads to https://hs-nix-template.cachix.org/.

[Skyfold]

@codygman, you are right! Though it seems the vim direnv could use a bit of polish. I think we can have the script as a backup for users without direnv editor integrations. However, the development process is much nicer with editor direnv integration. If you use lorri you don't have to wait for nix-shell to boot, so restarting gets much faster. Though you still have to watch for when your new shell is built to know when to reload ghcide.

[Skyfold]

@utdemir Thanks for setting that up!

[Skyfold]

Got another way we could setup ghcide to auto reload after lorri finishes a new build, see issue 371

[shapr]

At this point, I'd suggest adding haskell-language-server to the default packages

[utdemir]

At this point, I'd suggest adding haskell-language-server to the default packages

That makes sense to me, I just did that on master branch! It simply adds haskell-language-server binary (for the correct compiler version), so it can be invoked inside nix-shell.

Two points:

• I am not using a fancy editor, so I couldn't test the language server properly. It would be good if someone tests it and let me know if something else is necessary. (or send a PR). @shapr, could you?
• I don't know how editors can be set up to use the binary inside nix-shell. Presumably they will run something like nix-shell --run 'haskell-language-server --lsp, or alternatively we can set up something like direnv as mentioned by @codygman above, or any of the workarounds @Skyfold found. It would be useful if users of common editors (emacs, vi-likes or vscode comes to my mind) document their configurations.

[shapr]

I had occasion to test this today, and haskell-language-server is immediately available, thanks!

I'm using direnv with emacs, in case it helps any.

[utdemir]

I'm going to close this issue since haskell-language-server is now included and it seems to work.