hpack adds Paths_foo to other-modules without adding dependency on base

[josh-duetto]

hpack is adding Paths_* to the other-modules sections of executables (and tests?), but it's not adding a corresponding dependency on base. I'm seeing build failures when trying to compile the Paths_ module.

This is using stack 1.6.3/hpack 0.20.0 with the lts-10.3 resolver.

Example package.yaml:

name:                waka
version:             0.1.0.0
license:             BSD3

executables:
  waka-exe:
    main:                Main.hs
    dependencies:
      - protolude

Corresponding Main.hs:

{-# LANGUAGE NoImplicitPrelude #-}
{-# LANGUAGE OverloadedStrings #-}
module Main where

import Protolude

main :: IO ()
main = putStrLn ("ohai" :: Text)

Generated waka.cabal:

-- This file has been generated from package.yaml by hpack version 0.20.0.
--
-- see: https://github.com/sol/hpack
--
-- hash: a31ef4f39a7af096cd34ae4690cfca80fe95969a17c6e692f71c55a8bfdad95e

name:           waka
version:        0.1.0.0
license:        BSD3
build-type:     Simple
cabal-version:  >= 1.10

executable waka-exe
  main-is: Main.hs
  build-depends:
      protolude
  other-modules:
      Paths_waka
  default-language: Haskell2010

Build errors:

$ stack build
waka-0.1.0.0: configure (exe)
Configuring waka-0.1.0.0...
waka-0.1.0.0: build (exe)
Preprocessing executable 'waka-exe' for waka-0.1.0.0..
Building executable 'waka-exe' for waka-0.1.0.0..
[2 of 2] Compiling Paths_waka       ( .stack-work/dist/x86_64-osx/Cabal-2.0.1.0/build/waka-exe/autogen/Paths_waka.hs, .stack-work/dist/x86_64-osx/Cabal-2.0.1.0/build/waka-exe/waka-exe-tmp/Paths_waka.o )

/Users/josh/work/tmp/waka/.stack-work/dist/x86_64-osx/Cabal-2.0.1.0/build/waka-exe/autogen/Paths_waka.hs:10:1: error:
    Could not find module ‘Control.Exception’
    Use -v to see a list of the files searched for.
   |
10 | import qualified Control.Exception as Exception
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

/Users/josh/work/tmp/waka/.stack-work/dist/x86_64-osx/Cabal-2.0.1.0/build/waka-exe/autogen/Paths_waka.hs:11:1: error:
    Could not find module ‘Data.Version’
    Use -v to see a list of the files searched for.
   |
11 | import Data.Version (Version(..))
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

/Users/josh/work/tmp/waka/.stack-work/dist/x86_64-osx/Cabal-2.0.1.0/build/waka-exe/autogen/Paths_waka.hs:12:1: error:
    Could not find module ‘System.Environment’
    Use -v to see a list of the files searched for.
   |
12 | import System.Environment (getEnv)
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

/Users/josh/work/tmp/waka/.stack-work/dist/x86_64-osx/Cabal-2.0.1.0/build/waka-exe/autogen/Paths_waka.hs:13:1: error:
    Could not find module ‘Prelude’
    Use -v to see a list of the files searched for.
   |
13 | import Prelude
   | ^^^^^^^^^^^^^^


--  While building custom Setup.hs for package waka-0.1.0.0 using:
      /Users/josh/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2 --builddir=.stack-work/dist/x86_64-osx/Cabal-2.0.1.0 build exe:waka-exe --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1

It looks like either adding a dependency on base to package.yaml or removing Paths_waka from waka.cabal fixes the problem.

It seems like if hpack is introducing a dependency on base by automatically adding Paths_waka, it should also add the dependency.

[tfausak]

What version constraint should it put on base? Also, can you avoid the Paths_* module with other-modules: []?

[josh-duetto]

I don't know what constraints should be on base. I can see how tossing in surprise extra dependencies could cause problems. On the other hand, I'm not sure what the motivation is for adding the extra Paths_* module in the first place. I'm a fairly new Haskell dev and these build system details are still a bit beyond me at the moment.

And yes, I just tried specifying other-modules: [] in package.yaml, and that indeed suppresses the Paths_* addition, which leads to a successful build. Thanks! Is that the preferred work around?

[tfausak]

The constraints on base are only a problem because Hackage rejects uploads without upper bounds on base.

I’m not sure why the Paths_* module is automatically added. I think it’s always there, so it doesn’t really hurt anything. (Except obviously in this case.) I know I’ve had problems in the past with Paths_* because forgetting to add it to other-modules results in a linker error.

I don’t know if setting other-modules: [] is the preferred workaround, but it’s definitely a workaround. 🙂

[sol]

The general approach of Hpack is to assume reasonable defaults while still giving the user 100% control when the defaults don't match the situation.

Paths_* is added to other-modules because it is usually (except for in the situation at hand) not harmful and needed when the user wants to use its functionally. If we would not add it, the user would need to add it manually, loosing module inference for other-modules in the process.

The main motivation for adding it is: We want the user to benefit from module inference for other-modules even if he uses Paths_*.

With the upcoming generated-other-modules field, we could choose a different design, that is: We do not add Paths_* by default, a user then can add it to generated-other-modules when needed while still retaining module inference for other-modules. Is this a better solution? Not sure!

I don’t know if setting other-modules: [] is the preferred workaround, but it’s definitely a workaround. 🙂

It is the way to exercise 100% control over other-modules, which also results in loosing inference of other-modules. An other workaround is adding a base dependency manually.

[sol]

but it's not adding a corresponding dependency on base.

I'm not per se opposed to adding a dependency on base by default, as long as we stick to Hpack's general design principle (giving the user 100% control when needed). But I'm puzzled how the user would override this default, that is how would a user prevent the dependency on base if desired?

What version constraint should it put on base?

We could add an unrestricted dependency on base, at the point a user would want to upload to Hackage [s]he would have to specify it (her|him)self.

[cblp]

Yet another problem of adding Paths automagically:

/Users/cblp/dev/ff/<built-in>:15:10: error:
     error: non-portable path to file '".stack-work/dist/x86_64-osx/Cabal-2.0.1.0/build/FF/autogen/cabal_macros.h"'; specified path differs in case from file name on disk [-Werror,-Wnonportable-include-path]
#include ".stack-work/dist/x86_64-osx/Cabal-2.0.1.0/build/ff/autogen/cabal_macros.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         ".stack-work/dist/x86_64-osx/Cabal-2.0.1.0/build/FF/autogen/cabal_macros.h"
1 error generated.
`gcc' failed in phase `C pre-processor'. (Exit code: 1)

(I'm building on Mac with a case-insensitive FS.)

I don't know what is actually causing the problem here, maybe some bug in Cabal, but I don't see without Paths.

[tfausak]

It's a Cabal bug: haskell/cabal#4739

[cblp]

@tfausak thanks!

[sol]

It's a Cabal bug: haskell/cabal#4739

@tfausak @cblp Is this even related to Paths_* at all? Say would not adding Paths_* make this particular situation any better? Looks like this is related to cabal_macros.h. But I may be missing the full picture here.

[cblp]

@sol you are right, but my first thought was: why is Paths_*.hs even compiled?

[sol]

Closing in favor of #303. Feedback and suggestions are very much welcome!