Weird import behavior when using stack

[rpglover64]

Consider two simple files in a stack project:

FileA.hs:

module FileA where
import Data.List (foldl')

FileB.hs:

module FileB where
import Data.List
foo = foldl'

Running stack exec -- liquid [...] gives the following output:

% stack exec -- liquid FileA.hs
LiquidHaskell Copyright 2009-15 Regents of the University of California. All Rights Reserved.


**** DONE:  annotate ***********************************************************


**** ERROR: Invalid Source *****************************************************


 FileA.hs:2:19-24: Error: GHC Error
     Module ‘Data.List’ does not export ‘foldl'’

% stack exec -- liquid FileB.hs
LiquidHaskell Copyright 2009-15 Regents of the University of California. All Rights Reserved.


**** DONE:  annotate ***********************************************************


**** ERROR: Invalid Source *****************************************************


 FileB.hs:3:7-12: Error: GHC Error
     Not in scope: ‘foldl'’

As far as I can tell, liquid is opening the appropriate interface files (at least according to strace), but not using them properly (whatever that means)

[rpglover64]

There's a relevant IRC conversation here

(copied and cleaned up version):

20:20:58 <rpglover64> I'm having some issues with running liquidhaskell using `stack exec`.
20:21:28 <synergistics> shachaf: I got that, but I didn't see how s and t are themselves identities
20:21:32 <rpglover64> MyModule.hs:6:22-27: Error: GHC Error     Module ‘Data.List’ does not export ‘foldl'’
20:21:52 <rpglover64> which makes no sense to me
20:24:31 <rpglover64> (I do have an "import Data.List (foldl')" line in my file)
20:28:50 <tommd> rpglover64: You misunderstand the error.  It is telling you that Data.List does not define the function foldl', not that you aren't importing it.
20:29:05 <rpglover64> But Data.List does define foldl'
20:29:13 <tommd> rpglover64: Which isn't true for any distributed module named Data.List I know.
20:29:17 <tommd> Yes agreed.
20:29:30 <rpglover64> The error points to the import line
20:29:35 <tommd> rpglover64: So, do you have something in a Data directory called List.hs?
20:29:41 <rpglover64> no
20:29:43 <tommd> rpglover64: Yes I get that.
20:30:13 <jle`> rpglover64: what happens if you just imported Data.List?
20:30:55 <tommd> rpglover64: If you use ghci and `import Data.List` then `:type foldl'` what does it say?  From the same directory as you are compiling in, mind.
20:30:57 <rpglover64> jle`: error moves to use site: "Not in scope"
20:31:12 <jle`> referring to foldl' ?
20:31:13 <tommd> How are you compiling the code anyway?
20:31:14 <rpglover64> tommd: it gives me the type of foldl'
20:31:26 <rpglover64> stack exec -- liquid MyModule.hs
20:31:36 <rpglover64> jle`: yes
20:31:39 <jle`> hm, maybe try using the foldl' from Data.Foldable ?
20:31:39 <tommd> Ahh, liquid haskell?
20:31:41 <Axman6> that doesn't look right
20:31:58 <Axman6> at the very least, it should be stack exec liquid -- MyModule.hs
20:32:09 <Axman6> afaik anyway
20:32:34 <mgsloan> Either is fine
20:33:03 <Axman6> really? that's kinda gross =)
20:33:16 <tommd> I've happily used either before too.
20:34:17 <tommd> Makes me wonder what stack + liquid does in some cases.
20:35:40 <mgsloan> I don't know very much about liquid haskell, but I do know about stack.  So the ghci started by "stack exec ghci" gives the type of Data.List.foldl' ?
20:36:04 <mgsloan> AFAIK no version of Data.List ever lacked foldl', so I have no idea what's going on there, seems quite bizarre
20:36:16 <rpglover64> mgsloan: yes
20:36:28 <tommd> Yeah, liquid is the most suspicious thing here right now.
20:36:36 <rpglover64> I'm guessing liquid is failing to actually import the module
20:36:39 <rpglover64> somehow
20:36:57 <tommd> Perhaps the stack-ghc and library paths aren't being communicated.
20:37:53 <rpglover64> `stack exec env` seems to suggest that they are
20:38:33 <rpglover64> and the whole reason I started running `stack liquid` instead of bare `liquid` was to get access to `Data.Vector` (and it doesn't complain about missing the module)
20:38:47 <jle`> maybe try asking on the mailing list?
20:38:49 <jle`> it's fairly active
20:38:52 <tommd> Indeed, I can reproduce the issue using liquid.
20:38:58 <tommd> (and stack)
20:39:01 <rpglover64> That might be relevant; the behavior is the same for `Data.Vector`
20:39:10 <tommd> Using Liquid alone works fine.
20:43:08 <rpglover64> tommd: For reference, stracing confirms that it's finding the right interface files.
20:44:09 <tommd> rpglover64: OTOH, the "Module Imports:" output (--verbose) isn't present in the failing run with stack exec.
20:44:14 <tommd> It is there with just liquid.
20:44:58 <rpglover64> tommd: I think I'll file an issue on github and see where things go from there.

[gridaphobe]

Very strange.. These are both GHC errors, and we just pass a few -i flags to GHC. Does stack exec -- ghc FileA.hs work?

Also, are you using the latest hackage release or master? Thanks!

[gridaphobe]

Hrm, I can't reproduce this on master or 0.5.0.1.

% stack exec -- liquid FileA.hs
LiquidHaskell Copyright 2009-15 Regents of the University of California. All Rights Reserved.
[...]
**** RESULT: SAFE **************************************************************



% stack exec -- liquid FileB.hs
LiquidHaskell Copyright 2009-15 Regents of the University of California. All Rights Reserved.
[...]
**** RESULT: ERROR *************************************************************
Invalid Source

 FileB.hs:3:7-12: Error: GHC Error

 3 | foo = foldl'
           ^^^^^^

     No instance for (Foldable t0) arising from a use of ‘foldl'’

(My error in FileB is just the monomorphism restriction rearing its head)

[rpglover64]

I was using the latest hackage release (0.5.0.1). It's a shame the --version flag doesn't provide this info. stack exec -- ghc FileA.hs works.

What's especially weird is that I can reproduce this in exactly one stack directory. Even copies of that directory don't have the issue.

Is there a way I can see what flags are being passed to ghc?

[rpglover64]

...and the problem was using a liquid compiled with ghc-7.10.3 in a different stack project and stack installed in a ghc-7.10.2 project. Or at least, deleting the binary that was in my path fixed the problem.

[gridaphobe]

Is there a way I can see what flags are being passed to ghc?

We don't expose that with a flag, unfortunately, but the options we pass are essentially -i for every directory that shows up in paths (in verbose mode). We also have -g to pass flags through to GHC directly, i.e. -g "-package foo" would pass -package foo to GHC.

...and the problem was using a liquid compiled with ghc-7.10.3 in a different stack project and stack installed in a ghc-7.10.2 project. Or at least, deleting the binary that was in my path fixed the problem.

Interesting, we link against GHC instead of shelling out to the binary, so this could definitely cause problems if GHC changed something about the .hi format in the minor revision.

I'm not sure what we can do about this issue, maybe the ghc-mod or hdevtools folks have some experience here.

Thanks!

[rpglover64]

At least for ghc-mod, the answer is "You're on your own."

DanielG/ghc-mod#605

The solution I've found (for both) is to not run stack install ghc-mod, but to run stack build ghc-mod --compiler=ghc-7.10.2 outside of any stack sandbox (and similarly for ghc-7.10.3); then, the appropriate version gets added to a resolver-specific bin directory which gets added to the path on stack exec.

EDIT: this doesn't quite work... I have to also install it locally to the sandbox.

I've opened a stack issue: commercialhaskell/stack#1796

[gridaphobe]

I had a feeling this might be the case. Well, at the very least we should (hopefully) be able to check for a ghc-version mismatch and produce a more informative error message.

[gridaphobe]

@spinda could you investigate what it would take to produce a better error here? What I want to do is basically compare the version of GHC we compiled against to the version that produced the libraries we're loading. I think there's enough metadata around to do this.

[spinda]

Yes, this should be possible.

{-# LANGUAGE CPP #-}

compiledGhcVersion :: String
compiledGhcVersion = VERSION_ghc

getGhcApiVersion :: Ghc String
getGhcApiVersion = projectVersion <$> getSessionDynFlags

isCorrectGhcVersion :: Ghc Bool
isCorrectGhcVersion = (compiledGhcVersion ==) <$> getGhcApiVersion

[spinda]

@rpglover64 I'm having some trouble reproducing your environment. You built and stack install'd LiquidHaskell in an environment with ghc-7.10.3, then stack exec'd it in a project using ghc-7.10.2—right?

In that case, I don't understand how LiquidHaskell could end up reading .hi files compiled with ghc-7.10.2. LiquidHaskell would be linked against ghc-7.10.3 and, since LH produces its own copy of the .hi files every execution, it would be ghc-7.10.3 both creating and using in the .hi files, not the ghc-7.10.2 in the project environment.

Am I misunderstanding your situation?

[rpglover64]

You built and stack install'd LiquidHaskell in an environment with ghc-7.10.3, then stack exec'd it in a project using ghc-7.10.2—right?

Right.

In that case, I don't understand how LiquidHaskell could end up reading .hi files compiled with ghc-7.10.2. LiquidHaskell [...] produces its own copy of the .hi files every execution [...]

It doesn't produce .hi files for system libraries like Data.List, does it? That's where I think the issue is.

[rpglover64]

To reproduce:

1. stack exec -- ghc --version outside of a stack project (this issue crops up if I use a liquid from a different project too, but that's unnatural) to ensure it's ghc-7.10.3.
2. stack install liquidhaskell intern liquid-fixpoint
3. cd $THE_LIQUID_HASKELL_PROJECT
4. stack exec -- ghc --version to ensure it's ghc-7.10.2
5. stack exec which liquid to ensure that the binary exists and is the one created by stack install
6. stack exec liquid src/FileA.hs

If you'd like more detail on how to get the actual stack environments set up, let me know.

[spinda]

I see now. Thanks! So what needs to be checked, I think, is that the GHC version associated with the package DB matches the one we're expecting.