Enable tests whenever possible

[snoyberg]

See #805. Motivation: when you switch between cabal configure and cabal configure --enable-tests, the cabal_macros.h file is changed, leading to the library to be recompiled needlessly. Based on an idea from @gregwebs, this change makes it so that, whenever the test suite and benchmark dependencies are present, --enable-tests and --enable-benchmarks are passed in, respectively. We use components as a parameter to cabal build to control which components are built (unfortunately, there's no way to simply say "don't rebuild the library"...).

Pinging @chrisdone and @borsboom. @Fuuzetsu could you test out this branch and let me know if it helps out your stack build && stack test use case?

[gregwebs]

Thanks!

[chrisdone]

Hmm, what does this imply for this related discussion? #817 (comment) I'm suggesting that stack ghci match stack build by only loading libraries and executables by default, and then --test and --bench would enable those things. If this #815 issue is suggesting we build tests and benchmarks whenever they're available, I might as well leave GHCi's behaviour as-is. Thoughts?

[snoyberg]

It's not to build them, just configure them. This is nothing more than a
performance hack to work around weirdness in cabal. Semantics should remain
unchanged.

On Tue, Aug 18, 2015, 11:46 PM Chris Done [email protected] wrote:

Hmm, what does this imply for this related discussion? #817 (comment)
#817 (comment)
I'm suggesting that stack ghci match stack build by only loading
libraries and executables by default, and then --test and --bench would
enable those things. If this #815
#815 issue is suggesting
we build tests and benchmarks whenever they're available, I might as well
leave GHCi's behaviour as-is. Thoughts?

—
Reply to this email directly or view it on GitHub
#815 (comment)
.

[rpglover64]

Configuring them requires building their dependencies, though, right?

[snoyberg]

Right, which is why this checks if the dependencies are installed.

On Tue, Aug 18, 2015, 11:53 PM Alex R [email protected] wrote:

Configuring them requires building their dependencies, though, right?

—
Reply to this email directly or view it on GitHub
#815 (comment)
.

[chrisdone]

Right, then it ends up the same effect for stack ghci, I think, which is that it will make sure dependencies of test suites, for example, are available, and then configure ghci with the lib, exes, test suite and their dependencies. @rpglover64's concern is that stack ghci builds a bunch of extra packages for the test suite dependencies before loading up ghci.

[gregwebs]

I believe the patch only enables tests/benchmarks if the dependencies are present: it does not automatically build them until stack test, etc is actually invoked.

[Fuuzetsu]

It does not seem to work for my usecase…
First I run stack build twice to make sure everything builds fine. As expected second invocation does nothing. Then I run stack build --test which pulls in test dependencies and runs the tests. I run it again and I expect it to just run tests. It does not, it rebuilds my packages only and runs the tests. I run it yet again and this time it finally does only run tests.

So lastly I run stack build && stack build --test expecting it to first build nothing and then build nothing and just run the tests but instead it builds twice. Am I doing something wrong?

You can have a look at the project at tweag/HaskellR

[nix-shell:~/programming/HaskellR]$ stack build
ihaskell-inline-r-0.1.0.0-fcb209c0fe3bc1618e717bdd09f29de3: unregistering (missing dependencies: inline-r)
inline-r-0.6.0.1-ed2fb88558d5ba9e646ec512df7da2de: unregistering (local file changes)
inline-r-0.6.0.1: build
inline-r-0.6.0.1: install
H-0.6.0.1: configure
H-0.6.0.1: build
HaskellR-examples-0.1.0.0: configure
HaskellR-examples-0.1.0.0: build
H-0.6.0.1: install
ihaskell-inline-r-0.1.0.0: configure
ihaskell-inline-r-0.1.0.0: build
ihaskell-inline-r-0.1.0.0: install
HaskellR-examples-0.1.0.0: install
Completed all 4 actions.

[nix-shell:~/programming/HaskellR]$ stack build

[nix-shell:~/programming/HaskellR]$ stack build --test
ihaskell-inline-r-0.1.0.0-fcb209c0fe3bc1618e717bdd09f29de3: unregistering (missing dependencies: inline-r)
inline-r-0.6.0.1-ed2fb88558d5ba9e646ec512df7da2de: unregistering (components changed)
ieee754-0.7.6: download
ansi-wl-pprint-0.6.7.2: download
ansi-wl-pprint-0.6.7.2: configure
ansi-wl-pprint-0.6.7.2: build
ieee754-0.7.6: configure
ieee754-0.7.6: build
ansi-wl-pprint-0.6.7.2: install
inline-r-0.6.0.1: configure
inline-r-0.6.0.1: build
ieee754-0.7.6: install
optparse-applicative-0.11.0.2: download
optparse-applicative-0.11.0.2: configure
inline-r-0.6.0.1: install
optparse-applicative-0.11.0.2: build
H-0.6.0.1: configure
H-0.6.0.1: build
H-0.6.0.1: install
HaskellR-examples-0.1.0.0: build
optparse-applicative-0.11.0.2: install
ihaskell-inline-r-0.1.0.0: build
HaskellR-examples-0.1.0.0: install
quickcheck-assertions-0.2.0: download
quickcheck-assertions-0.2.0: configure
ihaskell-inline-r-0.1.0.0: install
quickcheck-assertions-0.2.0: build
quickcheck-assertions-0.2.0: install
regex-base-0.93.2: download
regex-base-0.93.2: configure
silently-1.2.4.1: download
regex-base-0.93.2: build
silently-1.2.4.1: configure
silently-1.2.4.1: build
silently-1.2.4.1: install
regex-base-0.93.2: install
tagged-0.7.3: download
tagged-0.7.3: configure
regex-tdfa-rc-1.1.8.3: download
tagged-0.7.3: build
regex-tdfa-rc-1.1.8.3: configure
regex-tdfa-rc-1.1.8.3: build
tagged-0.7.3: install
unbounded-delays-0.1.0.9: download
unbounded-delays-0.1.0.9: configure
unbounded-delays-0.1.0.9: build
unbounded-delays-0.1.0.9: install
regex-tdfa-rc-1.1.8.3: install
tasty-0.10.1.2: download
tasty-0.10.1.2: configure
tasty-0.10.1.2: build
tasty-0.10.1.2: install
tasty-hunit-0.9.2: download
tasty-hunit-0.9.2: configure
tasty-golden-2.3.0.1: download
tasty-hunit-0.9.2: build
tasty-golden-2.3.0.1: configure
tasty-golden-2.3.0.1: build
tasty-hunit-0.9.2: install
tasty-quickcheck-0.8.3.2: download
tasty-quickcheck-0.8.3.2: configure
tasty-golden-2.3.0.1: install
tasty-quickcheck-0.8.3.2: build
H-0.6.0.1: configure (test)
H-0.6.0.1: build (test)
tasty-quickcheck-0.8.3.2: install
inline-r-0.6.0.1: configure (test)
inline-r-0.6.0.1: build (test)
H-0.6.0.1: test (suite: tests)
inline-r-0.6.0.1: test (suite: test-qq)
inline-r-0.6.0.1: test (suite: test-shootout)
inline-r-0.6.0.1: test (suite: tests)
Completed all 19 actions.

[nix-shell:~/programming/HaskellR]$ stack build --test
ihaskell-inline-r-0.1.0.0-fcb209c0fe3bc1618e717bdd09f29de3: unregistering (missing dependencies: inline-r)
inline-r-0.6.0.1-ed2fb88558d5ba9e646ec512df7da2de: unregistering (local file changes)
inline-r-0.6.0.1: configure
inline-r-0.6.0.1: build
inline-r-0.6.0.1: install
H-0.6.0.1: configure
H-0.6.0.1: build
HaskellR-examples-0.1.0.0: configure
H-0.6.0.1: install
HaskellR-examples-0.1.0.0: build
H-0.6.0.1: configure (test)
H-0.6.0.1: build (test)
H-0.6.0.1: test (suite: tests)
ihaskell-inline-r-0.1.0.0: configure
ihaskell-inline-r-0.1.0.0: build
HaskellR-examples-0.1.0.0: install
inline-r-0.6.0.1: configure (test)
ihaskell-inline-r-0.1.0.0: install
inline-r-0.6.0.1: build (test)
inline-r-0.6.0.1: test (suite: test-qq)
inline-r-0.6.0.1: test (suite: test-shootout)
inline-r-0.6.0.1: test (suite: tests)
Completed all 6 actions.

[nix-shell:~/programming/HaskellR]$ stack build --test
inline-r-0.6.0.1: test (suite: test-qq)
H-0.6.0.1: test (suite: tests)
inline-r-0.6.0.1: test (suite: test-shootout)
inline-r-0.6.0.1: test (suite: tests)
Completed all 2 actions.

[nix-shell:~/programming/HaskellR]$ stack build && stack build --test
ihaskell-inline-r-0.1.0.0-fcb209c0fe3bc1618e717bdd09f29de3: unregistering (missing dependencies: inline-r)
inline-r-0.6.0.1-bb2ff6745c2711eb1b9b725ba28af388: unregistering (components changed)
inline-r-0.6.0.1: configure
inline-r-0.6.0.1: build
inline-r-0.6.0.1: install
H-0.6.0.1: configure
HaskellR-examples-0.1.0.0: build
H-0.6.0.1: build
H-0.6.0.1: install
ihaskell-inline-r-0.1.0.0: build
HaskellR-examples-0.1.0.0: install
ihaskell-inline-r-0.1.0.0: install
Completed all 4 actions.
ihaskell-inline-r-0.1.0.0-fcb209c0fe3bc1618e717bdd09f29de3: unregistering (missing dependencies: inline-r)
inline-r-0.6.0.1-bb2ff6745c2711eb1b9b725ba28af388: unregistering (components changed)
inline-r-0.6.0.1: configure
inline-r-0.6.0.1: build
inline-r-0.6.0.1: install
H-0.6.0.1: configure
HaskellR-examples-0.1.0.0: build
H-0.6.0.1: build
H-0.6.0.1: install
H-0.6.0.1: configure (test)
H-0.6.0.1: build (test)
HaskellR-examples-0.1.0.0: install
ihaskell-inline-r-0.1.0.0: build
H-0.6.0.1: test (suite: tests)
ihaskell-inline-r-0.1.0.0: install
inline-r-0.6.0.1: configure (test)
inline-r-0.6.0.1: build (test)
inline-r-0.6.0.1: test (suite: test-qq)
inline-r-0.6.0.1: test (suite: test-shootout)
inline-r-0.6.0.1: test (suite: tests)
Completed all 6 actions.

[snoyberg]

As I'm reading it, you're reporting that stack test followed by stack build recompiles with the message "components changed," correct? I ran into that issue earlier today and fixed it with 4cbca17. Does your compiled stack include that commit?

I'd recommend including --version output in each updated output for a report like this, since things are changing rather quickly.

[Fuuzetsu]

The stack I used for the above log was from this PR as I reported it, 8cbf096

I'll report back to you tomorrow with 4cbca17