"stack ghci" doesn't honor configured cpp flags

[peti]

The project https://github.com/peti/nix-paths uses Cabal's extension hooks to register additional flags for CPP in the build process. Both cabal-install and stack can build the project fine. The interactive REPL, however, works only for cabal-install. When I use stack to interpret the source code, I get the following error:

/tmp$ git clone git://github.com/peti/nix-paths.git
Cloning into 'nix-paths'...
remote: Counting objects: 34, done.        
remote: Compressing objects: 100% (21/21), done.        
remote: Total 34 (delta 10), reused 33 (delta 9), pack-reused 0        
Receiving objects: 100% (34/34), 6.14 KiB | 0 bytes/s, done.
Resolving deltas: 100% (10/10), done.
Checking connectivity... done.

/tmp$ cd nix-paths

/tmp/nix-paths$ stack init 
Writing default config file to: /tmp/nix-paths/stack.yaml
Basing on cabal files:
- /tmp/nix-paths/nix-paths.cabal

Checking against build plan lts-3.4
Selected resolver: lts-3.4
Wrote project config to: /tmp/nix-paths/stack.yaml

/tmp/nix-paths$ stack build 
nix-paths-1: configure
Configuring nix-paths-1...
nix-paths-1: build
Preprocessing library nix-paths-1...
[1 of 2] Compiling Nix.Paths        ( src/Nix/Paths.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Nix/Paths.o )
[2 of 2] Compiling Nix.FindFile     ( src/Nix/FindFile.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Nix/FindFile.o )
In-place registering nix-paths-1...
nix-paths-1: install
Installing library in
/tmp/nix-paths/.stack-work/install/x86_64-linux/lts-3.4/7.10.2/lib/x86_64-linux-ghc-7.10.2/nix-paths-1-CYictmtGnnPASIdBC8IhC0
Registering nix-paths-1...

/tmp/nix-paths$ stack ghci
nix-paths-1-75cbca45d7eb4d1f9747e6dd01e83080: unregistering (local file changes)
Configuring GHCi with the following packages: nix-paths
GHCi, version 7.10.2: http://www.haskell.org/ghc/  :? for help
[1 of 2] Compiling Nix.Paths        ( /tmp/nix-paths/src/Nix/Paths.hs, interpreted )

/tmp/nix-paths/src/Nix/Paths.hs:7:18:
    Not in scope: data constructor ‘PATH_TO_NIX_INSTANTIATE’

/tmp/nix-paths/src/Nix/Paths.hs:11:12:
    Not in scope: data constructor ‘PATH_TO_NIX_BUILD’

/tmp/nix-paths/src/Nix/Paths.hs:15:10:
    Not in scope: data constructor ‘PATH_TO_NIX_ENV’

/tmp/nix-paths/src/Nix/Paths.hs:19:12:
    Not in scope: data constructor ‘PATH_TO_NIX_STORE’
Failed, modules loaded: none.
Prelude> 

cabal-install, on the other hand, works just fine:

/tmp/nix-paths$ cabal repl
Package has never been configured. Configuring with default flags. If this
fails, please run configure manually.
Resolving dependencies...
[1 of 1] Compiling Main             ( dist/setup/setup.hs, dist/setup/Main.o )
Linking ./dist/setup/setup ...
Configuring nix-paths-1...
Preprocessing library nix-paths-1...
GHCi, version 7.10.2: http://www.haskell.org/ghc/  :? for help
[1 of 2] Compiling Nix.Paths        ( src/Nix/Paths.hs, interpreted )
[2 of 2] Compiling Nix.FindFile     ( src/Nix/FindFile.hs, interpreted )
Ok, modules loaded: Nix.Paths, Nix.FindFile.
*Nix.Paths> 

I suppose this means that stack doesn't correctly pick up the extra flags provided by Setup.lhs. I'm using Stack version 0.1.4.

[chrisdone]

Yeah, stack doesn't consult Setup.hs for flags at the moment.

[ranjitjhala]

Is there a way to pass the flags in at the command line ? (e.g. stack ghci --flags=...)? Thanks!

[snoyberg]

@ranjitjhala do you mean stack ghci --ghc-options="..."?

[peti]

Well, the problem IMHO is not so much passing the appropriate flags to stack but knowing what those appropriate flags are.

[snoyberg]

I was answering the previous question. I actually have no idea how Setup.hs generated flags work, and I'm not certain we have any way of passing them to ghci.

[mgsloan]

I've looked into this a bit, and the issue may actually be quite separate from the stack ghci. It looks to me like the configure hook lets you change pretty much anything about the package description. If you have it change the dependencies, does cabal-install use these new dependencies? What aspects of the package description can change?

If these things can change, then I'm not sure how we'd have full support of this. Stack does its whole build plan without running a single configure.

For this particular case, it works fine with stack build because the ./setup build that happens uses the dist/setup-config described here.

We might consider having a special case for stack ghci, though, and load up the package description from dist/setup-config.

[simonmichael]

I seem to be having the same problem with darcsden, which doesn't use cabal extension hooks. A flag is declared in the cabal file, enabled, and further down a conditional block sets a cpp flag. stack build sees the cpp flag but stack ghci doesn't. Even with today's stack from git and the --flag option.

$ darcs get http://hub.darcs.net/simon/darcsden
$ cd darcsden
$ stack build  # ok
$ stack ghci  # fails (RepoHandlers.hs:331:15:    Not in scope: ‘annotateFileHighlighting’)
$ stack ghci darcsden --flag darcsden:highlightingkate # fails

[mgsloan]

@simonmichael I think that's a different issue. Are you sure that you've got today's git version of stack? stack ghci works fine on darcsden, for me.

[peti]

@mgsloan,

If you have it change the dependencies, does cabal-install use these new dependencies?

No, the build plan is computed based on the information from the Cabal file.

What aspects of the package description can change?

I believe the readDesc hook gives you the ability to re-write the entire Cabal file as you please. Also, it's my understanding that Setup.hs is basically just any random executable, and it doesn't have to be based on Cabal at all.

If these things can change, then I'm not sure how we'd have full support of this. Stack does its whole build plan without running a single configure.

It is clearly impossible to have full support for any type of custom build. But that's fine, IMHO. Having support for those custom builds that do reasonable modification (like adding CPP flags, etc.) would be nice, though.

We might consider having a special case for stack ghci, though, and load up the package description from dist/setup-config.

Yes. All commands that run with a configured build environment available should take that information into account.

[mgsloan]

Cool, thanks for the info!

I believe the readDesc hook gives you the ability to re-write the entire Cabal file as you please

Gotcha. It looks to me like we can reasonably ignore all of the contents of LocalBuildInfo and just pay attention to localPkgDescr. This is nice, because that can be serialized as a .cabal file (I hope - showPackageDescription seems promising). I've added some notes here about how stack would do this: #1364

All commands that run with a configured build environment available should take that information into account.

Other than building Setup.hs files, this is the only place stack calls ghc directly, so it does make sense that we'd need to load up some info.

[crockeea]

What's the latest on this bug? My project is hitting it, and we'd love to get it fixed!

[mgsloan]

No news on this, I'd be glad to assist someone in addressing this, but probably can't fix the problem myself anytime soon, because it's pretty complex and we don't really need it. The state is pretty much as described in #1364 . That said, I would really like to close this gap in functionality for ghci.

I think ideally the way this would work is that custom-setup packages are forced to be loaded individually into ghci. Then we don't have to deal with merging the custom setup configured options with the options derived from the normal build type. At least, I'm assuming we'd get just some arguments to pass along to ghci. If we get a package description to use, then potentially these could be straightforwardly combined the info from other packages.

[cdepillabout]

I'm running into this limitation for termonad as well.

I'm defining new CPP macros in Setup.hs based on the version of the GTK system library (as discovered by pkg-config).

As described above, this works well for stack build, but it prevents the project from being loaded into GHCi with the stack ghci command.

Is there any sort of work around for getting this to work with stack?

[mihaimaruseac]

Can you try with stack exec -- ghci?

[cdepillabout]

@mihaimaruseac Thanks for the suggestion.

Unfortunately I'm not sure if stack exec -- ghci will work. I am setting new configuration options in Setup.hs. My problem is that unlike cabal repl, stack ghci does not run the Setup.hs to get those new configuration options.

I don't think stack exec -- ghci will execute Setup.hs, it will just launch ghci.

Maybe you meant stack exec -- cabal repl? I tried that too, just to see what would happen, but I get the following message:

$ stack exec -- cabal repl
cabal: Use of GHC's environment variable GHC_PACKAGE_PATH is incompatible with
Cabal. Use the flag --package-db to specify a package database (it can be used
multiple times).

[mihaimaruseac]

Yeah, I got into the same problem (i.e., stack not running Setup.hs) while trying to make haddock with internal libraries work.

So I guess the proper fix is to make stack run Setup.hs or simulate that

[jneira]

(From haskell/haskell-language-server#481)
I have a slightly different issue (although maybe the underneath cause is the same):

• i have a stack.yaml with

flags:
  stack-test:
    useide: true

• a .cabal file with

Flag useide
  description: whatever
  default: False

library:
  ....
  if flag(useide)
     cpp-options: -DUSEIDE

• a library module MyLib.hs with

someFunc :: IO ()
someFunc = do
#if USEIDE
    nonExistingDef
#else
    putStrLn $ "hello ide"
#endif

• stack build triggers correctly the error
• stack repl MyLib.hs --flag myPackage:useide triggers correctly the compile error
• stack repl MyLib.hs does not trigger it, not honouring the flag set in stack.yaml

Thanks!

[liskin]

@jneira This seems to be a slightly different issue as people here mentioned custom Setup.hs a lot whereas we're able to reproduce with a very plain cabal file. I filed a separate issue: #5434