Building tests can cause unnecessary recompilation

[crockeea]

The following steps cause the library portion of a project to be unnecessarily recompiled:

>stack build foo
>stack test foo

when the testsuite has a build-depend that isn't in the library.

For example, with this cabal file:

name:                foo
version:             0.1.1.0
stability:           experimental
build-type:          Simple
cabal-version:       >= 1.10

library
  default-language: Haskell2010
  exposed-modules:  Foo
  build-depends:    base
test-suite testfoo
  type:             exitcode-stdio-1.0
  default-language: Haskell2010
  main-is:          Main.hs
  build-depends:    base, data-default

I get the following:

> stack build foo
foo-0.1.1.0: configure
Configuring foo-0.1.1.0...
foo-0.1.1.0: build
Preprocessing library foo-0.1.1.0...
[1 of 1] Compiling Foo              ( Foo.hs, .stack-work/dist/x86_64-linux/Cabal-1.24.0.0/build/Foo.o )
foo-0.1.1.0: copy/register
Installing library in ...
Registering foo-0.1.1.0...

> stack test foo
**foo-0.1.1.0: unregistering (dependencies changed)**
foo-0.1.1.0: configure (lib + test)
Configuring foo-0.1.1.0...
foo-0.1.1.0: build (lib + test)
Preprocessing library foo-0.1.1.0...
Preprocessing test suite 'testfoo' for foo-0.1.1.0...
[1 of 1] Compiling Main             ( Main.hs, .stack-work/dist/x86_64-linux/Cabal-.24.0.0/build/testfoo/testfoo-tmp/Main.o )
Linking .stack-work/dist/x86_64-linux/Cabal-1.24.0.0/build/testfoo/testfoo ...
foo-0.1.1.0: copy/register

I get foo-0.1.1.0: unregistering (missing dependencies: data-default) when the additional dependency has not yet been build, or foo-0.1.1.0: unregistering (dependencies changed) if the additional dependency has already been built. There's no need to unregister the library if it has already been built and I would like to additionally build the tests.

[pepeiborra]

+1
We have worked around this in our Cabal files by aggregating the dependencies of all the stanzas in the library stanza, but this defeats Weeder (and common sense).

[mgsloan]

Unfortunately, this is due to behavior in Cabal that we cannot work around. When tests are enabled, it combines their dependencies. Worth investigating whether it has changed in Cabal-2.0, and opening an upstream issue about it if one doesn't already exist.

[kindaro]

@mgsloan  I experience the problem as described in this issue, but I can build the same test suite with Cabal and no spurious rebuilds happen. Is your last comment here still actual? Can you give me some evidence that I can go over to the Cabal folks with?

See also #4977.

[wraithm]

Yeah, honestly, this has been driving me insane for years.

[mgsloan]

@kindaro I don't know, I don't work on this stuff any more. I believe that newer Cabal versions treat components more homogenously and so this may indeed be resolveable.

There is some discussion of updating to Cabal's newer stuff in #2540

[wraithm]

I think this one is fixed, no?

[wraithm]

I think this one is fixed, no?

No, I confirmed that this issue is still there on master. Not fixed.

[wraithm]

Unfortunately, this is due to behavior in Cabal that we cannot work around. When tests are enabled, it combines their dependencies. Worth investigating whether it has changed in Cabal-2.0, and opening an upstream issue about it if one doesn't already exist.

@mgsloan Do you know if this is still the case? Is this behavior fixed in Cabal?

[mpilgrem]

This has come up recently on Matrix. I'll post more comprehensively here later. This is more a feature request than a bug: Stack is behaving as expected. Stack does not build a package's targeted components 'separately', it builds them 'all together' (if it can), so 'build lib against dependencies X, Y' and 'build lib (and test suite) against dependencies X, Y and Z' are different builds. There is a desire to change Stack so that it thinks of building in terms of 'components (of packages)' instead of 'packages (with components)', but that change is non-trivial. I think @theobat has been giving it the most thought in recent years, and he has an open issue here: #6356.

[wraithm]

@mpilgrem Awesome. Thank you for the summary here. It'd be amazing to have this feature. Do you know if this package-oriented limitation is still in the Cabal library? Or is it purely a stack issue? Said another way, given the state of Cabal today, is it theoretically possible to build this component-oriented build in stack?

[mpilgrem]

@wraithm, to approach building in terms of 'components (of packages)' requires Cabal >= 2.2, which is why Stack's support of Cabal < 2.2 was deprecated in Stack 2.15.1 and will be removed in a future version of Stack (it has already been removed from the master branch). As a consequence, future versions of Stack will not support GHC < 8.4 and, so, this decision was not taken lightly. It is discussed at: #6377.