Provide method for installing resolver-independent compiler-specific binaries

[rpglover64]

In particular, I'm thinking of ghc-mod (but liquid might also be in this boat).

stack exec sets the binary path to include the snapshot's bin directory and the compiler's bin directory (where things like ghc and haddock can be found). There is currently no high-level way to add a binary to all projects using the same compiler version; you could manually copy the binary to the same directory as ghc, but this doesn't seem like an ideal solution.

These binaries are typically installed for a user but, once compiled, care only which ghc version compiled it, not the snapshot it was compiled with.

[mgsloan]

This has been previously discussed here: #662

That said, generalizing the stuff stack setup does to install development tools beyond just msys / ghc / ghcjs could indeed be quite desirable. I don't see it as a big priority since a per snapshot install works ok (though is a bit wasteful)

[rpglover64]

Minor annoyances without this feature (using ghc-mod as the example):

• The naive approach of using stack install ghc-mod will lead to confusing error messages when stack exec -- ghc-mod is called in a snapshot with a different ghc version.
• Increased compile-times for all projects developers using ghc-mod (though the increase shouldn't be awful; there's probably a lot of shared work in common cases).
• Issues compiling newer versions of ghc-mod in snapshots using older resolvers. Sometimes these are as simple as adding a line to the stack.yaml, but in the worst case, ghc-mod might have dependency conflicts with other packages in the snapshot.
• Inability to use the stack.yaml that comes with ghc-mod.

As you say, it's not a big priority, since most users don't use multiple versions of the compiler in their development and can stack install, since per-snapshot installs are tolerable, and since it's not too difficult to manually copy/symlink the binary if that's really desired.

But it would be nice.

[mgsloan]

The naive approach of using stack install ghc-mod will lead to confusing error messages when stack exec -- ghc-mod is called in a snapshot with a different ghc version.

I consider that to be a ghc-mod bug, and I'm not sure why it still doesn't check ghc version.

Increased compile-times for all projects developers using ghc-mod (though the increase shouldn't be awful; there's probably a lot of shared work in common cases).

It should share the ghc-mod library, which is the majority of compilation time. So the main cost is the time to link the binary, and hard drive space for the binary.

Issues compiling newer versions of ghc-mod in snapshots using older resolvers. Sometimes these are as simple as adding a line to the stack.yaml, but in the worst case, ghc-mod might have dependency conflicts with other packages in the snapshot.

Fair point. A workaround is to compile ghc-mod with a resolver / extra-deps that work, and manually copy it into your project or snapshot's bin dir.

Inability to use the stack.yaml that comes with ghc-mod.

What's wrong with it? My understanding is that @DanielG does not regularly use stack to build ghc-mod, so the stack.yaml doesn't get maintained. ghc-mod should really run "stack build" on travis CI. Maybe open an issue on the ghc-mod tracker about it?

[rpglover64]

I consider that to be a ghc-mod bug, and I'm not sure why it still doesn't check ghc version.

This bug in particular. But I ran into what might be the same issue with Liquid Haskell (here), and I'm sure there will be more projects which rely on the GHC they're using being the same as the one used to compile them. It's fine to say that they all should check versions, but that doesn't save the hapless programmer who has to spend time debugging PATH issues.

It should share the ghc-mod library, which is the majority of compilation time.

I thought it could only do sharing if all dependencies are also shared, in which case there's a lot less sharing between distant lts versions.

A workaround is to compile ghc-mod with a resolver / extra-deps that work, and manually copy it into your project or snapshot's bin dir.

At that point, why not just copy it into the .stack/programs/x86_64-linux/ghc-7.10.3/bin directory?

What's wrong with it?

Nothing, but I can't use it if I already have a different stack.yaml file for my project and I need to compile ghc-mod in the same project to get the binary in the right place.

[mgsloan]

Yeah, there's also this PR - DanielG/ghc-mod#687 . I believe the reason it didn't get merged is that it'd add overhead to every ghc-mod invocation, which happens for every query. My proposed solution is to run the ghc version check whenever compilation fails. This may add a bit of overhead before ghc-mod exits, but hopefully the editor will display this output before it exits.

I thought it could only do sharing if all dependencies are also shared, in which case there's a lot less sharing between distant lts versions.

True! I suppose it is rather rare that you can avoid recompiling the ghc-mod library

At that point, why not just copy it into the .stack/programs/x86_64-linux/ghc-7.10.3/bin directory?

Valid! You could already do stack install ghc-mod --local-bin-path $(stack path --global-pkg-db)/../../../bin. Once #1546 is implemented, this could be simplified to stack install --local-bin-path $(stack path --ghc-path)/... Could have an alias like stack install-tool ghc-mod mean this. The main risk is confusing people about its meaning, which is really "install a development tool which isn't involved in the build process, but must match a particular ghc version"

EDIT: Ooh, based off the text of your issue, maybe stack install --compiler-specific ghc-mod would be good :)

[rpglover64]

stack install --compiler-specific ghc-mod sounds good to me! And you don't see any problems with putting more binaries in ghc-7.10.3/bin directory? Possibly create a new directory to decrease the impact of accidental (or malicious) name clashes? On the other hand, I've wanted to install that version of haddock that produces hyperlinked source since it was revealed...

[mgsloan]

Yeah, it's probably better to put things in a ghc-7.10.3/stack-tools/bin or something like that.

I wonder if it'd be best to put it in a separate .stack/tools/ folder (probably with essentially the same folder structure as .stack/programs. That way, if you want to start from scratch with all your installed ghc-specific tools, you can just blow away that one folder.

[DanielG]

@mgsloan the ghc-mod PR didn't get merged since it's unfinished and I haven't found the time to work on it, not because anything is wrong with it. Just FYI.

Probably a silly idea but how about looking at a binary's build-dependencies and when they include ghc automatically mark it as compiler specific or spit out a warning or something? Is that something you could do in stack?

[mgsloan]

@DanielG Interesting idea! Not all uses of ghc need to match the project's version, since they might not need the dist dir. However, in the name of consistency it does make sense to try to use binaries which use the exact same ghc version.

[DanielG]

We could add a field to the package's cabal file, something like x-stack-compiler-dependent: True instead to make it explicit. I'd just like something that doesn't require the user to do anything special.

[mgsloan]

Yeah, it'd be nice if it just "did the right thing" without the user thinking about it. Adding this degree of magic seems like it could potentially be confusing. It should come with a big notice in the output that "Hey I did something magical! Disable this magic with --no-magic". One question is whether this magic would still be in effect when --local-bin-path is specified.

[rpglover64]

--local-bin-path should tell stack where to put the binaries, regardless of other concerns. stack should probably warn or refuse to do a normal install on such binaries.

[mgsloan]

Hey all! Given that intero and other tools will benefit greatly from this, I have summarized our conclusions in #2643 , and so will close this discussion issue.

@rpglover64 I do not think --local-bin-path should override this. Feel free to comment on #2643